The  Effects  of  Slow  Frame  Rates  on  Human  Performance 


by  Jennifer  E.  Thropp  and  Jessie  Y.C.  Chen 


ARL-TR-3813 


June  2006 


Approved  for  public  release;  distribution  is  unlimited. 


NOTICES 

Disclaimers 

The  findings  in  this  report  are  not  to  be  construed  as  an  official  Department  of  the  Anny  position 
unless  so  designated  by  other  authorized  documents. 

Citation  of  manufacturer’s  or  trade  names  does  not  constitute  an  official  endorsement  or 
approval  of  the  use  thereof. 


DESTRUCTION  NOTICE — Destroy  this  report  when  it  is  no  longer  needed.  Do  not  return  it  to 
the  originator. 


Army  Research  Laboratory 

Aberdeen  Proving  Ground,  MD  21005-5425 


ARL-TR-3813 


June  2006 


The  Effects  of  Slow  Frame  Rates  on  Human  Performance 


Jennifer  E.  Thropp 
University  of  Central  Florida 

Jessie  Y.C.  Chen 

Human  Research  and  Engineering  Directorate,  ARL 


Approved  for  public  release;  distribution  is  unlimited. 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 
OMB  No.  0704-0188 


Public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and  maintaining 
the  data  needed,  and  completing  and  reviewing  the  collection  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information,  including  suggestions  for 
reducing  the  burden,  to  Department  of  Defense,  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports  (0704-0188),  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington, 
VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any  penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not 
display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1 .  REPORT  DATE  (DD-MM-YYYY) 

June  2006 


4.  TITLE  AND  SUBTITLE 


3.  DATES  COVERED  (From  -  To) 

May  2005  to  January  2006 


5a.  CONTRACT  NUMBER 


The  Effects  of  Slow  Frame  Rates  on  Human  Performance 


5b.  GRANT  NUMBER 


5c.  PROGRAM  ELEMENT  NUMBER 


6.  AUTHOR(S) 


Jennifer  E.  Thropp  (UCF);  Jessie  Y.C.  Chen  (ARL) 


5d.  PROJECT  NUMBER 

62716AH70 


5e.  TASK  NUMBER 


5f.  WORK  UNIT  NUMBER 


7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  Research  Laboratory 

Human  Research  and  Engineering  Directorate 

Aberdeen  Proving  Ground,  MD  21005-5425 


9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 


8.  PERFORMING  ORGANIZATION 
REPORT  NUMBER 

ARL-TR-3813 


10.  SPONSOR/MONITOR'S  ACRONYM(S) 


12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  is  unlimited. 


14.  ABSTRACT 


11.  SPONSOR/MONITOR'S  REPORT  NUMBER(S 


We  conducted  a  comprehensive  examination  of  the  effects  of  different  frame  rates  on  human  performance  and  reviewed 
more  than  50  studies  and  summarized  them  in  the  areas  of  psychomotor  performance,  perceptual  performance,  behavioral 
effects,  and  subjective  perception.  Overall,  there  seems  to  be  strong  support  for  a  threshold  of  around  15  Hz  for  many  tasks, 
including  those  that  are  psychomotor  and  perceptual  in  nature.  Less  impressive  yet  acceptable  performance  may  be 
accomplished  at  around  10  Hz  for  many  tasks.  Subjective  reactions  to  video  quality  seem  to  support  rates  of  5  Hz,  although 
videos  presented  at  1 5  Hz  and  above  are  generally  more  widely  preferred. 


15.  SUBJECT  TERMS 


degraded  video;  frame  rate;  human  performance 


16.  SECURITY  CLASSIFICATION  OF: 


a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

Unclassified  Unclassified  Unclassified 


17.  LIMITATION 

OF  ABSTRACT 

18.  NUMBER 

OF  PAGES 

SAR 

45 

19a.  NAME  OF  RESPONSIBLE  PERSON 

Jessie  Y.C.  Chen 


19b.  TELEPHONE  NUMBER  ( Include  area  code) 

407-384-5435 


Standard  Form  298  (Rev.  8/98) 
Prescribed  by  ANSI  Std.  Z39.18 


11 


Contents 


List  of  Figures  iv 

1.  Introduction  1 

1 . 1  Background . 1 

1.1.1  The  Mechanisms  of  FR . 2 

1.1.2  Related  Sources  of  Temporal  Distortion . 2 

1.1.3  Update  Rate  and  Refresh  Rate . 3 

1.1.4  Variable  Versus  Constant  FR . 4 

1 .2  Adverse  Effects  of  FR  on  Task  Performance . 4 

1 .2. 1  Task  Dependency . 5 

1 .2.2  Viewing  Condition  Dependency . 6 

1.2.3  Display  Luminance  and  the  Number  of  GLs . 7 

1 .2.4  Auditory  Cues  in  Addition  to  Visual  Cues . 7 

1 .2.5  End  User  Characteristics . 7 

2.  Psychomotor  Performance  8 

2. 1  Placement  Performance . 8 

2.1.1  Placement  Task  Conclusion . 9 

2.2  Tracking  Perfonnance . 9 

2.2. 1  Tracking  Task  Conclusion . 1 1 

3.  Perceptual  Performance  13 

3.1  Target  Recognition  Perfonnance . 13 

3.1.1  Constant  FR  Effects  of  T arget  Recognition . 13 

3.1.2  Variable  FR  Effects  of  Target  Recognition . 15 

3.1.3  Target  Recognition  Performance  Conclusion . 15 

3 .2  Perception  and  Psychophsics . 16 

3.2.1  Perception  and  Psychophsics  Conclusion . 17 

4.  Behavioral  Effects  19 

5.  Subjective  Assessment  19 


6.  Conclusions 


22 


7.  References 


25 


Appendix  A.  Summary  of  FR  Effects  on  Psychomotor  Performance  31 

Appendix  B.  Summary  of  FR  Effects  on  Perceptual  Performance  33 

Appendix  C.  Summary  of  FR  Effects  on  Subjective  Assessment  35 

Glossary  of  Acronyms  37 

Distribution  List  38 


List  of  Figures 

Figure  1.  Summary  of  FR  effects  on  psychomotor  performance . 12 

Figure  2.  Summary  of  FR  effects  on  perceptual  perfonnance . 18 

Figure  3.  Summary  of  FR  effects  on  subjective  assessment . 21 

Figure  4.  Summary  of  effects  of  slow  FR  (figures  1  through  3) . 22 


IV 


1.  Introduction 


1.1  Background 

In  recent  years,  there  have  been  growing  applications  of  robotic  technologies  in  fields  such  as 
space  exploration,  search  and  rescue,  national  defense,  entertainment,  police  special  weapons 
and  tactics  operations,  health  care,  and  personal  assistance  (Chen,  Haas,  Pillalamarri,  & 

Jacobson,  2006).  In  the  military  domain,  the  U.S.  Army’s  Future  Combat  System  (FCS) 
program,  for  the  first  time  in  the  U.S.  military  history,  envisions  a  future  battlefield  incorporating 
a  wide  array  of  unmanned  assets  including  aerial  and  ground  vehicles  as  well  as  unmanned 
sensor  platfonns  (Unit  of  Action  Maneuver  Battle  Lab,  2003).  These  unmanned  systems  will 
extend  the  ranges  and  capabilities  of  their  human  operators’  perception  and  action  and  will  have 
a  major  impact  on  future  combat  operations  (Oron-Gilad,  Chen,  &  Hancock,  2005).  In  order  to 
effectively  employ  the  unmanned  assets,  it  is  essential  that  the  robotic  operator  maintain  an 
effective  perception  of  the  remote  environment  through  the  communication  channel  between  the 
human  operator  and  the  robot.  However,  factors  such  as  distance,  obstacles,  bandwidth,  and 
electronic  jamming  may  pose  challenges  for  maintaining  sufficient  signal  strength  (French, 
Ghirardelli,  &  Swoboda,  2003).  As  a  result,  the  quality  of  video  feeds  that  a  robotic  operator 
relies  on  for  remote  perception  may  be  degraded  and  the  operator’s  performance  may  be 
compromised  (Van  Erp  &  Padmos,  2003). 

Common  forms  of  video  degradation  caused  by  low  bandwidth  include  reduced  frame  rate  (FR) 
(frames  per  second),  reduced  resolution  of  the  display  (pixels  per  frame),  and  a  lower  gray  scale 
(number  of  levels  of  brightness  or  bits  per  frame)  (Rastogi,  1996).  The  product  of  FR, 
resolution,  and  gray  scale  is  bandwidth  (bits  per  second),  and  it  is  important  to  determine  how  to 
balance  these  three  variables  with  a  given  bandwidth  so  that  operator  perfonnance  can  be 
optimized  (Sheridan,  1992).  Generally,  for  applications  in  virtual  environments  (VEs),  many 
researchers  recommend  10  Hz  (i.e.,  10  frames  per  second  or  10  fps)  to  be  the  minimum  FR  to 
avoid  performance  degradation  (Watson,  Walker,  Ribarsky,  &  Spaulding,  1998).  However, 
depending  on  the  nature  of  the  tasks,  the  same  degrees  of  slowing  of  FRs  may  have  different 
effects  on  human  performance.  To  provide  a  comprehensive  examination  of  the  effects  of 
different  FRs  on  human  performance,  this  report  reviewed  more  than  50  studies  and  summarized 
them  in  the  areas  of  psychomotor  perfonnance,  perceptual  performance,  behavioral  effects,  and 
subjective  perception.  First,  some  background  information  about  the  mechanisms  of  FR  and 
other  related  sources  of  temporal  distortions  are  presented.  This  discussion  is  followed  by  a  brief 
review  of  the  major  moderating  factors  that  can  impact  the  adverse  effects  of  slow  FR  on  human 
performance. 
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1.1.1  The  Mechanisms  of  FR 

FR  is  the  tempo  at  which  the  new  frames  of  the  graphical  scene  are  computed,  rendered,  and 
displayed.  Frame  time  (the  length  of  time  required  to  compute,  render,  and  display  the  image), 
which  is  the  inverse  of  FR,  is  the  primary  source  of  delay  in  desktop  personal  computers;  in  VEs, 
input  devices  such  as  three-dimensional  (3-D)  trackers  are  additional  sources  of  delay  (Watson, 
Walker,  Woytiuk,  &  Ribarsky,  2003).  FR  is  limited  by  several  computational  factors,  including 
frame  time,  which  is  then  determined  by  the  current  computational  demands  and  the  actual 
image  displayed  (i.e.,  complexity  or  level  of  detail  [LOD]).  Possible  sources  of  low  FR  in  VEs 
include  the  computation  required  to  process  tracker  data  (e.g.,  input  from  peripheral  devices  such 
as  mice  or  eye  trackers),  computation  speed  of  the  effects  of  user  movement  (e.g.,  use  of  head- 
mounted  displays  [HMDs]),  the  graphical  rendering  time,  and  communication  overhead  in 
distributed  systems.  If  any  of  these  factors  overwhelms  the  system  or  bandwidth  limitations,  the 
FR  may  be  reduced.  FR,  however,  is  not  an  entity  that  exists  in  isolation;  it  is  also  associated 
with  other  computational  demands. 

1.1.2  Related  Sources  of  Temporal  Distortion 

Bryson  (1993)  differentiates  FR  from  lag,  two  major  sources  of  temporal  distortion  in  real-time 
computing  systems.  Lag  is  the  temporal  discrepancy  between  the  data  stream  from  the  tracker 
(e.g.,  input  device  such  as  a  mouse)  and  the  actual  resulting  graphic  (Bryson,  1993).  Therefore, 
lag  is  the  time  delay  between  the  user’s  action  and  its  displayed  result  (Watson,  Spaulding, 
Walker,  &  Ribarsky,  1997).  In  terms  of  teleoperation,  delay  is  considered  to  be  the  temporal 
difference  between  communication  between  the  local  and  remote  sites  (i.e.,  how  long  it  takes  for 
the  user’s  command  or  input  to  see  a  result  in  the  system’s  action)  (Tharp,  Liu,  French,  Lai,  & 
Stark,  1992).  Lag  is  also  referred  to  as  delay,  latency,  and  system  responsiveness  (SR). 

Lag  is  created  by  the  following  factors:  delays  in  tracker  signal,  delays  in  communication 
between  tracker  and  computer,  delays  from  computations  involved  in  processing  tracker  data, 
and  delays  from  graphical  rendering.  SR  resolution,  however,  can  be  complicated.  SR  is  the 
combination  of  system  latency  (SL)  and  the  time  between  the  completion  of  the  user’s  last  action 
and  the  next  source  of  input  into  the  display.  Furthermore,  Watson  et  al.  (1997)  explain  that 
frame  time,  SL,  and  SR  vary  randomly.  SL  is  the  frame  time  plus  time  needed  for  the  system  to 
collect  input  samples  from  the  real  world,  such  as  tracker  perfonnance.  The  variation  in  latency 
is  greater  than  the  variation  in  frame  time,  and  the  variation  in  SR  is  the  greatest,  as  it  builds 
upon  the  variation  in  both  frame  time  and  SL.  Thus,  it  is  important  to  consider  the  means  and 
the  standard  deviations  of  these  display  speeds. 

Watson  et  al.  (2003)  maintain  that  there  are  several  factors  that  cause  delay  in  an  interactive 
graphics  system,  some  of  which  affect  only  frame  time,  while  others  impact  only  latency,  and 
still  others  impact  both  frame  time  and  latency.  However,  all  these  potential  sources  impact  SR. 
Thus,  FR  and  lag  are  closely  related  and  are  often  manipulated  at  the  same  time  (e.g.,  see  Ware 
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&  Balakrishnan,  1994).  Bryson  (1993)  has  stated  that  it  is  difficult  to  isolate  the  effects  of  lag 
and  FR  from  other  types  of  VE  distortions.  Watson  et  al.  (1997)  also  state  that  lag  is  closely 
associated  with  FR  and  that  they  should  both  be  considered  by  designers.  However,  there  is  not 
much  research  to  guide  these  designers. 

Latency  and  FR  occur  together  but  must  be  carefully  manipulated  in  experiments  in  order  for 
effects  to  be  properly  detected.  Ellis,  Adelstein,  Baumeler,  Jense,  and  Jacoby  (1999)  found  that 
latency  was  a  more  reliable  and  stronger  influence  than  FR  on  tracking  performance  in  a  VE.  In 
a  tracing  task,  latency  was  also  more  influential,  having  a  multiplicative  effect  on  response  time 
(Arthur,  Booth,  &  Ware,  1993).  This  may  be  because  latency  can  be  experimentally  varied  over 
a  greater  range  than  FR.  For  instance,  in  a  study  by  Ellis  et  al.  (1999),  experimental  latency 
ranged  from  80  to  480  ms,  yielding  a  proportion  of  1:6,  whereas  the  FR  range  of  6  to  20  fps 
yields  a  proportion  of  1:3.7. 

1,1.3  Update  Rate  and  Refresh  Rate 

Update  rate  in  teleoperation  or  in  VE  applications  refers  to  the  frequency  at  which  the  image  of 
the  remote  site  is  captured  and  then  displayed  to  the  remote  operator  (e.g.,  via  HMD  or  other 
displays),  and  it  depends  on  the  bandwidth  limitation  (Liu,  Tharp,  French,  Lai,  &  Stark,  1993). 
Update  rate,  therefore,  is  the  upper  limit  and  detenninant  of  display  rate.  For  instance,  if  the  FR 
of  the  HMD  peaks  at  100  Hz,  and  the  update  rate  was  30  Hz,  images  could  only  be  displayed  at  a 
maximum  rate  of  30  Hz.  Usually,  in  the  VE  applications,  FRs  are  higher  than  the  update  rates, 
so  the  image  would  appear  smooth.  In  some  VE  literature,  however,  update  rate  and  FR  are 
sometimes  used  interchangeably  (Kolasinski,  1995). 

Refresh  rate,  on  the  other  hand,  refers  to  the  number  of  images  presented  to  the  eye  every  second 
(usually  between  60  and  10  Hz)  (Richard,  Birebent,  Coiffet,  Burden,  Gomex,  &  Langrana, 

1996).  Basically,  refresh  rate  is  a  hardware-determined  constant,  while  update  rate  can  vary 
widely  based  on  scene  complexity  and  other  factors  such  as  available  computing  power  for 
generating  the  images  (Kolasinski,  1995;  Pausch,  Crea,  &  Conway,  1992).  The  following 
example  by  Richard  et  al.  helps  to  clarify  the  distinction  between  refresh  rate  and  FR:  a  display 
system  with  a  refresh  rate  of  60  Hz  and  an  FR  of  4  Hz  will  present  1 5  consecutive  identical 
images  before  there  are  any  changes  in  the  scene. 

Watson  et  al.  (1998)  state  that  high  LOD  in  a  frame’s  image  and  adequate  SR  facilitate  user 
performance,  yet  make  strong  computational  demands  that  often  end  in  an  LOD/SR  trade-off. 
Update  rate  is  directly  proportional  to  bandwidth  and  is  impacted  by  factors  such  as  computation 
and  rendering  speeds  of  the  graphics  system.  Thus,  update  rate  could  be  increased  if  scenes  were 
generated  with  simpler  components  such  as  lines  and  bioptic  images  as  opposed  to  polygons  and 
stereo  images  (Liu  et  al.,  1993).  Cai  (2004)  describes  the  merit  of  context-aware  displays,  in 
which  the  human  user’s  perceptual  state  (e.g.,  user’s  scanning  behavior  with  context 
anticipation)  and  the  actual  display  content  are  monitored  in  order  to  provide  the  optimum  LOD 
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for  that  specific  image.  This  is  an  important  consideration  since  human  visual  search  can  be 
guided  by  content  and  purpose  (Yarbus,  1967),  and  excessive  LOD  can  be  minimized  when  not 
needed  in  order  to  allocate  bandwidth  to  other  demands,  such  as  higher  FR.  In  terms  of  temporal 
resolution,  Cai  (2004)  found  that  when  video  was  presented  at  less  than  4  Hz,  bandwidth  usage 
was  clearly  lower  than  continuous  streaming  at  approximately  17  Hz. 

1.1.4  Variable  Versus  Constant  FR 

Many  bandwidth  studies  have  empirically  manipulated  constant  FRs.  Yet  the  amplitude  and 
deviation  of  FRs  can  vary  in  a  computing  system,  ultimately  yielding  a  variable  FR  which  may 
be  reported  as  a  range  or  mean.  This  variability  can  occur  in  presentations  with  significant 
change  in  the  detail  between  scenes  in  the  frames.  Normally,  in  VEs,  an  adaptive  detail 
management  system  adjusts  the  object  polygon  count  from  frame  to  frame,  depending  on  the 
LOD  needed,  and  consequently,  this  determines  the  frame  update  rate  in  order  to  present  an 
approximately  constant  polygon  count  in  each  scene.  However,  when  the  LOD  changes 
dramatically  between  frames,  an  adaptive  detail  management  system  may  initially  calculate  the 
timing  of  the  next  frame  erroneously. 

Fluctuations  in  the  actual  frame  times  occur  even  when  a  specific  FR  is  sought  in  a  controlled 
environment.  In  other  words,  a  particular  FR  can  actually  be  a  range  of  FRs,  which  yields  a 
mean  FR  along  with  standard  deviations  in  frame  time.  For  instance,  Watson  et  al.  (1997)  used 
the  experimental  stimuli  presented  for  a  100-ms  frame  time  and  introduced  a  standard  deviation 
in  frame  time  of  20  ms,  which  in  turn  generated  a  range  of  frame  times  of  72  to  129  ms  (the 
inverse  of  which  corresponds  to  a  range  of  FRs  of  13.9  to  7.8  Hz).  The  implementation  of  frame 
time  standard  deviations  can  replicate  the  reality  that  numerous  mechanisms  result  in  a  variable 
rather  than  constant  FR  in  video  transmission.  Procedures  for  calculating  variable  FR  values  are 
presented  in  detail  in  Watson  et  al.  (1997). 

1.2  Adverse  Effects  of  FR  on  Task  Performance 

Extremely  low  update  rates  can  be  problematic.  As  the  FR  becomes  lower,  the  time  (lag) 
between  frames  becomes  longer,  and  the  scene  may  appear  jittery.  Objects  may  consequently 
appear  to  move  in  saccades,  and  the  human  visual  system  has  to  conduct  spatiotemporal 
interpolation  to  “fill”  these  visual  gaps  (Richard  et  al.,  1996).  Arthur  et  al.  (1993)  state  that 
frame  update  rate  and  the  lag  in  receiving  and  processing  tracker  data  are  among  the  two  factors 
that  are  often  associated  with  human  performance  in  virtual  worlds.  Head-coupled  tracking 
involves  tracking  head  movements  so  that  when  the  user  moves  his  or  her  head,  the  perspective 
generated  on  the  display  will  mimic  real-life  changes  in  perspective  from  head  movements. 
Arthur  et  al.  state  that  FR  is  directly  related  to  a  specific  type  of  head-coupled  display  system  lag 
in  which  there  is  a  delay  between  eye  position  reception  and  display  update.  This  is  the  time  to 
compute  and  render  a  scene  from  the  user’s  perspective,  since  current  tracker  measurements  are 
needed;  two  are  needed  if  the  display  is  stereoscopic. 
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Tharp  et  al.  (1992)  and  Liu  et  al.  (1993)  state  that  3-D  tracking  and  “pick-and-place”  tasks  are 
two  skills  that  are  often  required  for  tele-manipulation.  Reddy  (1997)  explains  that  poor  FR  and 
large  delay  can  degrade  human-system  interaction,  often  in  terms  of  impaired  depth  perception 
on  the  visual  display.  Thus,  the  use  of  low  FRs  can  have  adverse  effects  on  tele-manipulation, 
especially  as  far  as  depth  perception  is  concerned.  However,  there  are  several  moderating  factors 
that  might  impact  the  adverse  effects  of  low  FRs.  In  this  section,  we  briefly  review  five  of  those 
moderators:  task  dependency,  viewing  condition  dependency,  display  luminance  and  the  number 
of  gray  scale  levels  (GL),  auditory  cues  in  addition  to  visual  cues,  and  end  user  characteristics. 

1.2.1  Task  Dependency 

Johnson  and  Caird  (1996),  who  investigated  the  impact  of  FR  on  sign  language  gesture 
recognition,  remind  readers  that  the  required  FR  for  perfonnance  of  a  task  can  depend  on  the 
nature  of  the  task  itself.  If  animation  must  be  conveyed,  then  considering  that  the  human  visual 
systems  takes  50  msec  to  process  a  frame,  20  Hz  may  be  required  (Card,  Moran,  &  Newell, 

1983).  However,  conveying  a  basic  understanding  of  a  sequence  may  require  fewer  frames. 

It  has  been  reported  that  10  Hz  is  the  minimum  threshold  for  performance  in  immersive  VEs  (see 
Bryson,  1993;  MacKenzie  &  Ware,  1993;  Watson  et  al.,  1997,  1998).  However,  higher  rates  of 
10  to  15  Hz  are  required  for  acceptable  performance  (McKenna  &  Zeltzer,  1992).  For 
architectural  walk-through  tasks,  various  minimum  FRs  have  been  recommended,  including  6  Hz 
(Airey,  Rohlf,  &  Brooks,  1990),  7  Hz  (Pausch,  1991),  and  10  Hz  (Bryson,  1993;  Card, 

Robertson,  &  MacKinlay,  1991;  McKenna  &  Zeltzer,  1992). 

Watson  et  al.  (2003)  differentiate  between  open  and  closed  loop  tasks,  each  of  which  is  affected 
by  FR.  In  a  closed  loop  task,  such  as  tracking,  continuous  feedback  is  provided.  Thus,  the 
result  of  the  previous  action  feeds  into  the  next  action.  Continuous  feedback  is  therefore 
required  for  the  operator  to  execute  the  next  move.  Automobile  driving  is  considered  to  be  a 
closed  loop  tracking  task  because  the  driver  is  required  to  maintain  position  of  the  vehicle  within 
the  specified  lanes  and  to  follow  a  prescribed  pathway.  In  a  tracking  task,  delay  increases  the 
time  that  passes  until  feedback  is  received  by  the  user,  and  as  a  result,  performance  can  degrade. 
Difficult  tracking  tasks  are  further  impaired  by  delays. 

Open  loop  tasks,  on  the  other  hand,  have  little  or  no  feedback  available  to  the  operator.  For 
example,  placement  task  performance  depends  on  predictive  planning  and  is  not  guided  by 
adjustments  of  feedback  as  are  closed  loop  tracking  tasks.  However,  placement  tasks  end  in  a 
closed  loop  phase  where  finer  adjustments  are  made  to  place  the  object  in  its  final  position, 
which  is  a  small  target  space. 
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1.2.2  Viewing  Condition  Dependency 

Parkhurst  and  Niebur  (2001)  demonstrate  the  human  observer’s  different  needs  of  visual  detail 
for  different  viewing  conditions.  In  a  velocity-based  LOD  graphics  rendering  technique,  objects 
remain  stationary  but  appear  as  a  blur  in  the  observer’s  visual  field  if  he  or  she  turns  his  or  her 
head.  Therefore,  the  blurred  object  does  not  need  to  have  a  high  LOD.  Thus,  LOD  is  reduced 
as  the  object  moves  across  the  visual  field  but  then  increases  again  as  the  observer  slows  and 
stops  his  or  her  viewpoint  rotation.  In  this  rendering  method,  LOD  is  linearly  related  to 
rotational  velocity  of  the  observer.  This  method  also  keeps  the  system’s  computational  load 
roughly  constant. 

In  distance-based  LOD  manipulation  (i.e.,  real-time  viewpoint-dependent  simplification),  objects 
that  are  farther  away  from  the  viewer  do  not  need  to  be  rendered  at  high  complexity.  This 
reduction  in  detail  then  allows  more  resources  to  be  allocated  to  the  generation  of  higher  FRs,  as 
per  the  LOD-FR  trade-off.  There  are  other  times  when  the  human  visual  system  does  not 
naturally  process  high  LOD,  such  as  when  objects  are  in  the  periphery  of  the  visual  field  (Virsu 
&  Rovamo,  1979)  and  when  rapidly  moving  objects  appear  as  blurred  (Murphy,  1978;  Burr  & 
Ross,  1982). 

Both  pixel  size  and  FR  can  influence  the  accuracy  of  perception  of  motion  in  depth.  Even 
though  the  object  may  be  moving  at  a  constant  velocity,  this  velocity  appears  to  accelerate  or 
decelerate,  depending  on  whether  it  is  located  near  or  far  from  the  viewer.  According  to  Pfautz 
(2002),  the  motion  of  small  objects  near  the  line  of  sight  and  far  from  the  viewer  is  likely  to  be 
limited  by  pixel  size.  However,  the  motion  of  large  objects  far  from  the  line  of  sight  and  near  the 
viewer  is  likely  to  be  limited  by  FR.  If  FR  and  pixel  size  are  correct,  then  the  viewer  can  detect 
perspective  depth. 

There  may  also  be  differential  effects  for  stereoscopic  and  monoscopic  displays  (see  Arthur  et 
al.,  1993;  Lion,  1993).  In  monoscopic  displays,  the  image  is  presented  to  only  one  eye;  in  some 
settings,  this  can  be  done  to  avoid  retinal  image  disparity.  In  stereoscopic  displays,  both  eyes 
receive  an  image,  but  the  image  displayed  to  each  eye  is  slightly  different  because  of  the  position 
of  the  eyes  relative  to  each  other.  Lion  (1993)  found  that  stereo  perfonnance  was  significantly 
better  than  mono  performance  in  a  manual  tracking  task.  This  was  confirmed  by  Richard  et  al. 
(1996),  who  demonstrated  that  tracking  task  completion  time  was  lower  overall  for  the  stereo 
condition. 

There  may  also  be  effects  for  the  use  of  head-coupled  imagery  on  visual  performance  (see  Lion, 
1993).  In  restricted  head  coupling,  participants  rest  their  chins  against  a  chin  rest  to  control  for 
the  effects  of  parallax.  This  is  another  viewing  condition  variable  which  may  be  a  moderator  in 
the  relationship  between  FR  and  human  performance  if  both  are  manipulated  simultaneously. 
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1.2.3  Display  Luminance  and  the  Number  of  GLs 

The  number  of  GLs  is  a  characteristic  of  the  video  monitor  and  is  usually  measured  in  bits. 
Whiting,  Honig,  Carterette,  and  Eigler  (1991)  demonstrated  a  linear  relationship  between  GL  and 
video  display  luminance:  in  the  range  of  80  to  180  gl,  there  is  a  slope  of  0.209  (candela 
[cd]/m")/GL.  Whiting  et  al.  were  able  to  generate  a  mean  luminance  of  6.2  cd/m  (1.8  ft-lumen), 
which  corresponds  to  a  GL  of  128.  GL  can  also  serve  as  a  source  of  image  noise;  Whiting  et  al. 
manipulated  pixel  noise  standard  deviations  of  5,  10,  14,  and  20  GL  for  static  images  and  20  GL 
for  dynamic  noise.  Also,  image  contrast  can  be  defined  as  the  GL  increment  of  the  target.  Thus, 
GL  could  be  a  characteristic  of  visual  display  quality  that  could  impact  target  detection 
performance  independently  as  well  as  interactively  with  FR  (also  see  Vitkovitch  &  Barber, 

1994). 

1.2.4  Auditory  Cues  in  Addition  to  Visual  Cues 

It  is  also  necessary  to  consider  the  influence  of  auditory  components  of  visual  presentations.  In 
some  settings,  audition  is  thought  to  play  a  larger  role  in  temporal  perception  than  vision  (e.g., 
see  Welch,  Duttonhurt,  &  Warren,  1986).  For  instance,  musical  tempo  can  influence  the  human 
perception  of  time  in  passing  and  display  FR  (Mastoropoulou  &  Chalmers,  2004; 

Mastoropoulou,  Debattista,  Chalmers,  &  Troscianco,  2005).  Speech  reading  ability,  which  is 
rooted  in  target  recognition,  is  significantly  more  accurate  in  audio-visual  conditions  than  in  an 
audio-only  mode  (Vitkovitch  &  Barber,  1994;  Frowein  et  al.,  1991).  This  effect  may  be  a 
moderator  that  should  be  distinguished  from  possible  FR  manipulation  effects. 

Audiovisual  skew  or  asynchrony  between  the  video  and  the  audio  aspects  of  the  image  sequence 
can  also  affect  recognition.  It  can  be  caused  by  low  FRs  that  present  obsolete  visual  information 
that  does  not  match  the  continuous  real-time  delivery  of  sound.  While  this  mismatch  can 
typically  impair  performance,  it  can  be  helpful  in  some  instances  of  low  FR.  For  example,  an 
asynchrony  in  which  audio  lags  the  visual  component  of  the  presentation  can  actually  aid 
performance  beyond  that  which  can  be  normally  perceived  at  30  Hz  (Knoche,  deMeer,  &  Kirsh, 
2005).  The  direction  of  audio-visual  skew  can  be  a  moderator  worth  considering.  Therefore,  it 
is  important  to  examine  the  full  range  of  effects  generated  by  an  interaction  among  audition, 
vision,  and  FR. 

1.2.5  End  User  Characteristics 

Although  many  aspects  of  the  system  and  experimental  methods  can  influence  the  relationship 
between  FR  and  performance,  it  is  important  to  consider  the  individual  differences  in  the  humans 
who  use  these  systems.  One  possible  moderator  is  the  level  of  experience  the  operator  has  with 
computers  and  virtual  reality  (VR).  Those  who  have  been  exposed  to  these  systems  are  more 
familiar  with  the  different  viewing  and  psychomotor  effects  that  can  result  from  temporal 
distortions  which  would  otherwise  not  be  present  in  the  everyday,  real-world  human-machine 
experience.  Many  of  today’s  computing  and  simulation  environments  are  capable  of  rendering 
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graphics  at  30  Hz  and  higher.  Operators  of  these  advanced  systems  may  therefore  be 
unaccustomed  to  the  effects  of  low  FR  and  lag.  For  instance,  Liu  et  al.  (1993)  suggest  a 
minimum  update  rate  of  2  Hz  for  experienced  users  and  10  Hz  for  inexperienced  users  of  a 
teleoperation  tracking  task.  Adaptation  to  poor  temporal  fidelity  has  also  been  demonstrated. 
Increasing  the  operators’  exposure  to  low  temporal  fidelity  can  aid  performance,  and  training  is 
one  way  to  accomplish  adaptation  (see  Johnson  &  Caird,  1996). 


2.  Psychomotor  Performance 


2.1  Placement  Performance 

According  to  Fitts’  Law,  the  time  required  to  acquire  a  target  is  a  function  of  (a)  the  physical 
distance  to  the  target  and  (b)  the  physical  size  of  the  target  (Fitts,  1954).  The  interaction  between 
low  FR  on  Fitts’  Law  was  assessed  by  Bryson  (1993,  Experiment  2).  This  placement  task 
involved  moving  a  cursor  from  an  initial  position  to  a  final  position  in  a  computer  system.  The 
performance  means  showed  that  increased  lag  and  low  FR  have  similarly  detrimental  effects  on 
placement  performance.  More  specifically,  lags  and  FRs  greater  than  250  ms  (<  4  Hz) 
dramatically  increased  the  difficulty  level  of  the  task.  Furthermore,  the  effect  of  great  lag  and 
low  FR  was  less  pronounced  in  the  participant  who  was  experienced  with  VEs  with  poor  SR.  In 
addition,  Massimino  and  Sheridan  (1994)  suggested  that  frame  rate  as  low  as  5  Hz  might  be 
viable  for  placement  tasks  if  force  feedback  were  available. 

In  Liu  et  al.  (1993),  operators  performed  pick-and-place  tasks  while  wearing  HMDs.  They  found 
that  operators  who  were  experienced  with  HMDs  were  able  to  perform  the  placement  tasks  at 
update  rates  as  low  as  10  Hz  but  had  significantly  more  errors  below  2  Hz.  Those  who  were 
inexperienced  demonstrated  impairments  once  the  update  rate  dropped  below  30  Hz,  which 
suggests  that  experience  level  can  moderate  the  relationship  between  FR  and  performance.  Liu 
et  al.  also  observed  that  task  completion  time  was  longer  at  update  rates  below  10  Hz. 

Meehan,  Insko,  Whitton,  and  Brooks  (2002)  assessed  placement  perfonnance  in  VEs;  however, 
they  also  measured  the  physiological  reactions  imposed  by  various  FRs.  Lower  FRs  can  impair 
performance  and  feelings  of  presence.  For  instance,  the  10-Hz  condition  caused  many  participants 
to  lose  a  sense  of  balance,  causing  anomalous  dependent  measure  scores  which  consequently 
required  readjustment.  The  change  in  heart  rate  was  significantly  higher  in  the  10-Hz  than  in  the 
15-Hz  condition,  indicating  increased  stress  at  the  lower  FR.  Reported  behavioral  presence 
significantly  increased  as  FR  increased  from  15  to  20  Hz  and  from  20  to  30  Hz. 

Watson  et  al.  (1997)  examined  placement  performance  in  the  context  of  variable  FRs,  using  the 
sinusoidal  method  to  calculate  standard  deviations  in  frame  time.  Average  placement  and  grab 
times  significantly  improved  when  mean  FR  increased  from  10  Hz  to  20  Hz.  In  a  later  study, 
Watson  et  al.  (1998)  again  assessed  the  effects  of  FR  means  and  standard  deviations  on  grasping 
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and  placement  tasks.  This  time,  standard  deviations  were  calculated  with  the  frame-latency 
manipulation  method  and  were  absolute  values  of  FRs  (e.g.,  4  Hz).  Grasp  time  and  the  number 
of  grasp  attempts  were  significantly  greater  at  the  9-Hz  mean  FR  and  when  the  standard 
deviation  of  system  responsiveness  (SDSR)  was  4  Hz.  Overall,  grasping  performance  was 
impaired  the  most  when  mean  FRs  were  low  and  SDs  were  high.  Based  on  these  results,  Watson 
et  al.  (1998)  speculated  that  improving  the  mean  FR  above  17  Hz  may  not  necessarily  increase 
grasping  performance;  to  test  this,  a  second  study  was  conducted  with  a  set  of  higher  FRs.  FR 
SDs  were  based  on  percentages  of  FRs  (e.g.,  5.6%  of  17  Hz).  It  was  found  that  increasing  FR 
from  17  to  25  Hz  did  not  significantly  improve  grasping  perfonnance,  as  hypothesized. 

Improved  placement  performance  occurred  at  higher  mean  FRs  and  lower  standard  deviations,  as 
was  also  found  in  Experiment  1 .  Overall,  these  results  suggest  that  there  may  be  a  grasping 
performance  threshold  at  higher  FRs  (such  as  at  17  Hz)  and  that  grasping  performance  may  be 
compromised  only  at  lower  mean  FRs  (such  as  9  and  13  Hz). 

Methods  of  calculating  SDs  in  frame  time  have  also  been  compared.  In  a  third  experiment, 
Watson  et  al.  (1998)  tested  the  hypothesis  that  SDSR  control  via  absolute  values  was  inferior  to 
SDSR  control  via  percentages  of  mean  FR.  The  SDs  were  manipulated  as  absolute  values  of 
FRs  (e.g.,  4  Hz).  Results  showed  that  there  was  a  significant  increase  in  grasp  time  only  when 
the  FR  was  at  the  lowest  (17  Hz)  and  SDSR  was  highest  (7.56  Hz).  A  reduction  in  the  SR  range 
occurred  at  higher  FRs,  and  therefore  absolute  FR  control  did  not  yield  many  significant  effects. 
Thus,  Watson  et  al.  suspected  that  SR  control  based  on  absolute  FR  values  may  be  inferior  to  SR 
control  based  on  percentages  of  mean  FRs. 

2,1.1  Placement  Task  Conclusion 

Results  from  studies  that  have  manipulated  a  constant  FR  indicate  that  10  Hz  and  less  can  be 
detrimental  to  placement  perfonnance  in  a  VE;  higher  FRs  are  needed  for  participant 
performance  and  comfort.  Specifically,  the  research  has  indicated  that  FRs  less  than  4  Hz  seem 
to  dramatically  increase  the  difficulty  level  of  the  task  (Bryson,  1993).  Also,  participants  in  VEs 
experience  discomfort,  balance  loss,  and  increased  heart  rate  at  10  Hz  (Meehan  et  al.,  2002). 
However,  the  variable  FR  approach  may  be  more  accurate  when  FRs  are  low,  since  standard 
deviations  and  fluctuations  of  FRs  are  important  considerations  (Watson  et  al.,  1997).  FR 
thresholds  for  ideal  perfonnance  in  variable  FR  configurations  are  probably  similar  to  constant 
FR  thresholds.  Watson  et  al.  (1997)  found  that  grasping  performance  was  compromised  only  at 
lower  mean  FRs  such  as  the  9  and  13  Hz.  For  improved  placement  perfonnance,  the  average  FR 
should  increase  to  at  least  17.5  Hz  (Watson  et  al.,  1998)  or  20  Hz  (Watson  et  al.,  1997). 

2.2  Tracking  Performance 

Bryson  (1993,  Experiment  1)  assessed  tracking  perfonnance  at  various  FRs  and  found  that  in 
general,  nonnalized  error  was  linearly  dependent  on  frame  time.  Furthermore,  the  slope  of  this 
linear  relationship  depended  on  the  frequency  of  the  targets  moving  in  a  sinusoidal  path.  Thus, 
longer  frame  times  (i.e.,  lower  FRs)  at  higher  frequencies  of  target  motion  produced  the  greatest 
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tracking  error.  In  consideration  of  these  results  and  those  from  Experiment  1  (see  placement 
performance  section),  Bryson  (1993,  Experiment  1)  concluded  that  frame  time  effects  are 
congruent  with  those  of  delay  in  tracking  tasks.  Cai  (2004)  found  that  human  figure  recognition 
and  tracking  in  a  video  were  found  to  be  a  logarithmic  function  of  FR,  and  a  major  improvement 
in  this  performance  occurs  by  increasing  FR  from  0.05  Hz  to  1  Hz.  Thus,  the  minimal 
satisfactory  FR  was  1  Hz  in  normal  viewing  conditions.  Performance  increases  somewhat 
between  1  and  7  Hz  and  then  seems  to  stabilize  from  7  Hz  until  the  maximum  tested  value  of 
17  Hz. 

Ware  and  Balakrishnan  (1994)  assessed  the  effects  of  low  FR  as  well  as  lag  in  a  target 
acquisition  in  VR  display.  Reaction  time  reductions  were  the  most  dramatic  until  5  Hz  but 
stabilized  at  around  10  Hz,  beyond  which,  perfonnance  improvements  are  minimal.  Thus,  it  may 
not  be  worth  the  extra  cost  to  increase  FRs  beyond  10  Hz  for  this  type  of  task  and  setting. 

Vehicle  control  is  considered  to  be  a  tracking  task  since  the  driver  must  maintain  position  of  the 
vehicle  within  the  lanes  following  a  specific  pathway  (Watson  et  al.,  2003).  In  a  study  of 
teleoperation  of  unmanned  ground  vehicles  (UGVs),  McGovern  (1991)  did  not  find  driving 
performance  degradation  when  image  update  rates  were  lowered  from  30  to  7.5  Hz.  On  the  other 
hand,  Van  Erp  and  Padmos  (2003)  reported  that,  when  update  rate  dropped  below  5  Hz  in  an 
indirect  driving  task,  participants’  lateral  control  significantly  degraded.  Day  (1999)  also 
assessed  the  effects  of  low  FRs  on  a  remote  vehicle  driving  task.  Mean  task  completion  times 
and  mean  time  in  error  increased  as  FR  decreased,  and  performance  for  FRs  1  through  4  Hz  was 
significantly  lower  than  for  the  control  condition,  25  Hz.  Not  surprisingly,  post-task 
questionnaires  indicated  that  50%  of  participants  felt  they  would  have  performed  tasks 
significantly  better  with  higher  quality  video. 

Teleoperation-based  tracking  performance  is  of  particular  importance  in  FR  studies,  as 
communication  problems  can  arise  from  bandwidth  limitation  (Fiu  et  al.,  1993).  The  UGV 
operator  faces  possible  sources  of  signal  degradation  in  the  forms  of  physical  distance  from  the 
UGV,  obstacles,  and  electronic  jamming  (French  et  al.,  2003).  Successful  teleoperation 
therefore  relies  on  tracking  perfonnance  in  terms  of  navigation  as  well  as  target  tracking.  Fiu  et 
al.  found  that,  similar  to  their  placement  task  performance  previously  discussed,  operators  who 
were  experienced  with  HMDs  were  able  to  perfonn  tracking  tasks  at  update  rates  as  low  as 
10  Hz  but  had  significantly  more  errors  below  2  Hz.  Inexperienced  operators,  on  the  other  hand, 
showed  a  performance  degradation  almost  immediately  when  FRs  started  to  decrease.  French  et 
al.  found  that  low  FRs  (i.e.,  2  and  4  Hz)  yielded  significantly  longer  navigation  times  and  higher 
cognitive,  visual,  and  subjective  workload  ratings,  but  not  target  identification  or  situational 
awareness.  French  et  al.  concluded  that  16  Hz  is  ideal  for  UGV  teleoperation.  However,  to 
obtain  effective  task  performance,  a  minimum  of  8  Hz  is  necessary. 
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Display  format  and  viewing  characteristics  can  also  interact  with  FR  to  impact  manual  tracking 
performance.  Lion  (1993)  considered  the  differential  effects  for  stereoscopic  and  monoscopic 
displays,  as  well  as  the  use  of  head-coupled  imagery. 

The  33-Hz  FR  yielded  better  manual  tracking  perfonnance  than  the  22-Hz  FR.  The  cursor  was 
also  kept  closer  to  the  target  21%  more  of  the  time  in  the  33-Hz  condition  as  opposed  to  the 
22-Hz  condition.  There  was  also  a  significant  interaction  between  stereo  displays  and  FR,  with 
better  perfonnance  in  the  stereo  33-Hz  condition. 

An  interaction  between  viewing  condition  and  FR  was  found  by  Richard  et  al.  (1996).  Tracking 
time  was  fastest  and  stable  in  the  range  of  14  Hz  to  28  Hz  in  the  mono  condition  but  remained 
stable  at  FRs  as  low  as  7  Hz  in  the  stereo  condition.  Stereo  vision  also  continued  to  improve 
performance  much  more  than  that  of  mono  vision  when  the  FR  dropped  below  7  Hz.  After  10 
trials  at  1  to  7  Hz  in  the  mono  condition,  the  mean  target  capture  time  produced  a  flat  curve, 
suggesting  cessation  of  learning.  However,  in  the  stereo  condition,  the  mean  capture  time  was 
lower  overall,  and  the  curve  appeared  mostly  flat  from  the  earliest  trials.  Thus,  less  learning  took 
place  at  1  to  7  Hz  in  the  stereo  condition. 

Lampton,  Knerr,  Goldberg,  Bliss,  Moshell,  and  Blau  (1994)  observed  that  FR  degradations  affect 
participants’  tracking  performance  in  a  VE.  Although  the  FRs  were  not  systematically  manipu¬ 
lated,  Lampton  et  al.  did  find  that  the  slow  FRs  (i.e.,  approximately  5  Hz)  made  the  3-D  tracking 
tasks  very  difficult.  It  was  reported  that  participants  were  able  to  keep  the  cursor  on  the  moving 
target  less  than  9%  of  the  time. 

Ellis  et  al.  (1999)  examined  tracking  performance  in  VEs  at  various  FRs  and  degrees  of  latency. 
Decreased  FR  (e.g.,  6  Hz  versus  20  Hz)  significantly  increased  root  mean  square  (rms)  error, 
increased  perceived  control,  and  decreased  feelings  of  stability.  Participants’  judged  realism  was 
not  affected  by  the  various  FR  and  latency  values,  however. 

Arthur  et  al.  (1993)  manipulated  the  effects  of  stereo  versus  mono  displays  as  well  as  the  use  of 
head-coupled  imagery.  Error  rates  in  a  3-D  tree-tracing  task  were  measured.  Response  time 
increased  along  with  increased  lag  and  the  lower  FR  of  10  Hz.  Regression  equations  indicated 
that  lag  likely  contributed  more  to  performance  effects  than  did  reduction  in  FR. 

2.2.1  Tracking  Task  Conclusion 

Overall,  higher  FRs  such  as  20  Hz  and  33  Hz  yielded  better  manual  tracking  perfonnance  than 
lower  FRs  such  as  6  Hz  and  below  (see  Ellis  et  al.,  1999;  Lion,  1993).  The  detrimental  effect  of 
low  FR  is  especially  notable  when  the  target  moved  at  higher  frequencies  (Bryson,  1993, 
Experiment  1). 
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Teleoperation  may  require  higher  FRs.  French  et  al.  (2003)  suggested  that  16  Hz  is  ideal  for 
tracking  in  UGV  teleoperation,  and  that  8  Hz  is  an  absolute  minimum  requirement.  Additionally, 
Day  (1999)  demonstrated  that  remote  vehicle  control  was  significantly  better  at  25  Hz  than  at 
4  Hz. 


Acceptable  tracking  performance  was  found  to  be  possible  at  very  low  FRs.  Rates  above  10  Hz 
may  not  be  necessary  for  some  applications;  thus,  systems  with  smaller  bandwidth  capability 
may  still  be  effective.  The  minimal  satisfactory  FR  was  1  Hz  for  tracking  a  moving  human 
figure  (Cai,  2004)  and  2  Hz  to  10  Hz  for  teleoperation  for  experienced  operators  (Liu  et  al., 
1993).  Finally,  target  acquisition  in  a  3-D  environment  seems  to  benefit  most  when  FRs  are 
increased  to  10  Hz  (Ware  &  Balakrishnan,  1994). 


There  may  also  be  a  significant  interaction  between  stereo  and  FR,  with  better  performance  in 
stereo  at  higher  FRs.  It  seems,  therefore,  that  the  combination  of  high  FRs  and  stereo  displays 
more  closely  mimics  the  real-world  visual  experience,  thereby  requiring  less  user  adaptation  and 
learning  (Richard  et  al.,  1996).  This  suggests  that  overall,  if  lower  refresh  rates  are  used, 
stereoscopic  displays  may  compensate  to  boost  operator  performance.  Based  on  these  results, 
Lion  (1993)  proposes  the  importance  of  giving  design  priority  to  the  development  of  systems 
with  stereoscopic  displays  and  higher  FR. 


A  summary  of  these  research  findings  is  presented  graphically  in  figure  1  and  is  also  provided  in 
more  detail  in  appendix  A. 


Summary  of  Frame  Rate  Effects  on  Psychomotor 

Performance 
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(Note:  Pink  squares  signify  the  FR  level  where  human  performance  starts  to  significantly  degrade.  Blue 
diamonds  signify  the  lowest  FR  levels  where  participants  perfonn  normally.) 


Figure  1.  Summary  of  FR  effects  on  psychomotor  performance. 
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3.  Perceptual  Performance 


3.1  Target  Recognition  Performance 

Some  researchers  examined  the  effects  of  slow  FR  on  target  recognition  performance  in  the 
military  context.  For  example,  French  et  al.  (2003)  investigated  robotic  operators’  target 
identification  perfonnance  using  a  UGV.  Their  simulated  degradation  of  video  feed  from  the 
UGV  included  2,  4,  8,  and  16  Hz.  They  found  that  FRs  as  slow  as  2  Hz  did  not  significantly 
affect  their  perfonnance,  including  target  identification  and  situational  awareness. 

Chen,  Durlach,  Sloan,  and  Bowens  (2005)  conducted  a  robotic  control  study  in  a  simulated 
military  environment.  They  manipulated  the  FRs  (5  versus  30  Hz)  of  the  sensor  video  feed  of 
the  simulated  unmanned  aerial  vehicle  and  UGV.  Participants’  tasks  included  target  detection 
and  acquisition.  Neither  performance  was  significantly  degraded  as  FRs  dropped  from  30  to 
5  Hz. 

Perception  of  the  visual  cues  associated  with  speech  and  lip  reading  may  also  be  considered  a 
form  of  target  recognition.  Visual  information  can  aid  interpersonal  communication  for  those 
with  impaired  hearing  by  means  that  coupled  the  visual  cues  of  speech  reading  or  “lip  reading” 
with  the  limited  auditory  perception  that  remains.  Sign  language  interpretation  is  a  pattern 
recognition  task  which  may  depend  on  temporal  resolution.  FR  is  thought  to  impact  specific 
aspects  of  lip  reading,  including  the  apparent  duration  of  visual  speech  components  such  as 
mouth  movements  (Massaro,  Beskow,  Cohen,  Fry,  &  Rodriguez,  1999)  and  dynamic 
components  such  as  syllable  timing  (Summerfield,  1979). 

3.1.1  Constant  FR  Effects  of  Target  Recognition 

Johnson  and  Caird  (1996)  found  that  low  FR  can  impair  recognition  of  American  Sign  Language 
gestures.  Although  there  was  no  significant  main  effect  for  FR,  means  indicated  a  trend  for 
impaired  performance  and  increased  error  rate  at  lower  FRs  (e.g.,  1  and  5  Hz).  However,  over 
the  course  of  additional  trials,  there  was  an  improvement  in  performance.  Training  may 
therefore  be  useful  when  bandwidth  is  limited. 

Adding  audio  cues  to  visual  presentation  can  aid  target  recognition.  Frowein  et  al.  (1991) 
assessed  speech  reception  target  recognition  in  the  domain  of  video  telephony.  Audio-visual 
recordings  of  common  sentences  were  presented  in  both  audio-visual  and  audio-only  modes. 
Participants  were  asked  to  repeat  each  sentence  immediately  after  the  presentation.  At  5  to  6  Hz, 
performance  in  the  audiovisual  condition  was  significantly  better  than  the  audio  condition,  and  at 
15  Hz,  the  added  perfonnance  improvement  stabilized,  with  no  further  performance  benefits  for 
the  30-Hz  condition.  Thus,  there  may  be  an  interaction  between  presentation  mode  and  FR  in 
speech  perception. 
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Vitkovitch  and  Barber  (1994)  hypothesized  that  the  perception  of  the  speaker’s  facial  movement 
was  important  to  help  listeners  to  perceive  speech.  In  this  study,  participants  listened  to  two 
simultaneous  recordings  of  a  message  and  shadowed  (i.e.,  repeated)  the  words  of  the  target 
message.  Recordings  were  in  either  audio-only  or  audio-visual  format.  Having  a  visual  display 
did  improve  performance  over  an  audio-only  display.  Even  audio-visual  video  presented  at  the 
lowest  FR  (8.3  Hz)  yielded  improved  perfonnance  over  the  audio-only  condition;  this  is 
consistent  with  Frowein  et  al.  (1991).  Across  all  FRs,  performance  improved  when  FR  was 
increased.  However,  there  was  no  significant  performance  benefit  from  increasing  the  FR  from 
16.7  Hz  to  25  Hz,  which  suggests  that  16.7  Hz  may  be  adequate  for  video  transmission  for 
recognition  tasks  of  this  nature;  this  is  also  consistent  with  Frowein  et  al. 

In  another  study  by  Vitkovitch  and  Barber  (1996,  Experiment  1)  participants  lip  read  number 
sequences  in  the  range  of  1  through  30  in  a  visual-only  condition.  By  turning  off  the  sound, 
Vitkovitch  and  Barber  (1996)  were  able  to  determine  that  the  detrimental  effect  of  low  FR  is  not 
attributable  to  audiovisual  asynchrony.  GF  of  the  image  was  also  manipulated.  Performance 
accuracy  significantly  improved  when  FRs  were  increased  from  8.3  Hz  to  25  and  from  8.3  Hz  to 
16.7.  There  was  no  significant  gray  scale  and  FR  interaction,  which  suggests  that  both  may  be 
independent  (yet  additive)  sources  of  visual  cues  in  lip  reading  ability.  In  a  second  experiment, 
Vitkovitch  and  Barber  (1996)  sought  to  assess  whether  repeated  exposures  to  stimuli  of  low  FR 
and  limited  gray  scales  can  aid  participants’  lip  reading  ability.  Both  digital  and  analog  formats 
as  well  as  GF  were  compared.  There  was  no  improvement  in  accuracy  over  time  for  those  in  the 
low  FR  (8.3  Hz).  This  may  indicate  that  the  high  perceptual  demand  of  the  low  FR  was  too 
great,  and  thus  increased  experience  was  not  enough  to  allow  participants  to  compensate  for  the 
visual  degradation.  Main  effects  of  both  FR  and  GFs  indicate  that  performance  was  higher  at 
25  Hz  than  8.3  Hz  as  well  as  16  GF  compared  to  8  GF.  Results  of  both  experiments  show  that 
FRs  higher  than  8.3  Hz  are  necessary  for  lip  reading  with  16.7  Hz  being  adequate. 

Williams,  Rutledge,  Garstecki,  and  Katsaggelos  (1997)  investigated  the  effects  of  FR  and  the 
recognition  of  visemes  or  visual  phonemes  (Fisher,  1968),  which  are  the  visual  speech  cues  such 
as  mouth  movements  and  teeth  positions  that  accompany  speech  sounds.  Generally,  viseme 
recognition  decreased  along  with  decreasing  FR.  Often,  viseme  recognition  dramatically 
dropped  when  the  FR  was  reduced  from  5  Hz  to  2  Hz. 

The  visual  distortion  caused  by  low  FR  can  result  in  an  asynchrony  with  the  audio  components 
of  the  presentation.  Knoche  et  al.  (2005)  manipulated  both  FR  and  audiovisual  skew  upon  the 
ability  of  participants  to  identify  the  middle  consonants  of  nonsense  words.  Results  showed  that 
lower  FRs  worsen  the  effects  of  negative  audio  skew  (in  which  audio  leads  visual).  Conversely, 
positive  skews  (in  which  visual  leads  audio)  can  improve  consonant  comprehension.  At  FRs 
between  10  and  15  Hz,  audio  lags  between  +127  ms  and  +167  ms  can  improve  comprehension 
beyond  that  which  normally  occurs  at  30  Hz.  This  may  be  because  longer  frame  times  allowed 
participants  to  “close  the  gap”  between  audio  and  visual  stimuli.  Additionally,  the  authors 
suspect  there  is  a  ±120-ms  limit  on  the  ability  for  humans  to  integrate  audiovisual  infonnation 
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(any  more  or  less  suggests  great  asynchrony  and  possible  failed  integration).  Whereas  audio  and 
video  can  be  perceived  in  synchronization  with  each  other  anywhere  between  ±120  ms  and 
30  Hz,  this  window  changes  to  -54  ms  to  ±186  ms  at  10  Hz.  The  implications  of  these  results  are 
that  in  a  “noisy”  environment,  video  at  10  Hz  will  be  effective  if  audio  lags  visual  by  +120  ms  to 
+  170  ms. 

FR  studies  can  also  apply  to  dynamic  medical  displays,  such  as  x-ray  fluoroscopy,  cardiac 
cineangiography,  real-time  two-dimensional  (2-D)  ultrasound,  rapid  sequence  nuclear  magnetic 
resonance  imaging  (MRI),  radioisotope  ventriculography,  and  ultrafast  computer  tomography. 
There  is  also  a  degree  of  “noise”  and  contrast  involved  in  these  displays,  which  can  further 
complicate  the  detection  of  low  contrast  signals.  Whiting  et  al.  (1991)  examined  the  effects  of 
FR  on  signal  detection  in  a  radiological  application.  Higher  FRs  aided  signal  detection 
performance,  specifically  as  the  FR  progressed  from  0  Hz  (static  image)  to  approximately  4  Hz. 

It  then  improved  modestly  until  the  FR  increased  to  16  Hz,  then  mildly  improved  at  higher  FRs 
thereafter.  Although  100  to  200  msec  was  previously  suggested  as  the  maximum  persistence  of 
visual  memory,  the  results  of  current  study  suggest  a  1500-msec  decay  time  in  this  task. 

3.1.2  Variable  FR  Effects  of  Target  Recognition 

Variable  FR  configurations  have  been  tested  in  target  recognition.  Parkhurst  and  Niebur  (2001) 
employed  the  velocity-based  LOD  graphics  rendering  technique  to  present  target  objects  in  a 
virtual  room.  FRs  were  held  constant  at  6  Hz  or  were  variable  so  that  they  were  allowed  to 
increase  with  decreasing  LOD  as  the  observer  turned  his  or  her  head  to  search  for  targets.  Target 
detection  accuracy  did  not  change  because  of  LOD  changes.  Reaction  times  increased  with 
decreasing  LOD  in  the  constant  FR  condition,  but  in  the  variable  FR  condition,  reaction  times 
decreased  when  LOD  decreased,  likely  because  the  extra  bandwidth  was  allocated  toward 
generating  a  higher  FR.  Overall,  there  was  support  for  the  velocity-based  LOD  rendering 
technique,  since  faster  rendering  times  associated  with  decreased  LOD  can  aid  visual  search 
performance. 

3.1.3  Target  Recognition  Performance  Conclusion 

Some  researchers  examined  the  effects  of  slow  FR  on  target  recognition  performance  in  the 
military  context.  For  example,  French  et  al.  (2003)  investigated  robotic  operators’  target 
identification  performance  and  found  that  FRs  as  slow  as  2  Hz  did  not  significantly  affect  their 
performance  (as  compared  to  when  FRs  were  4,  8,  and  16  Hz).  Similarly  Chen  et  al.  (2005)  did 
not  observe  significant  target  detection  performance  degradation  as  FRs  dropped  from  30  to 
5  Hz. 

Speech  reading  is  one  form  of  target  recognition.  Frowein  et  al.  (1991)  concluded  that  FR  is  an 
important  contributor  to  speech  reading  perception  and  that  FRs  until  15  Hz  can  improve  this 
performance.  Comparably,  Vitkovitch  and  Barber  (1994,  1996)  concluded  that  16.7  Hz  may  be 
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adequate,  and  lower  rates,  such  as  8.3  Hz  to  12.5  Hz,  may  result  in  a  great  loss  of  dynamic  visual 
information  and  audio-visual  mismatch,  thereby  causing  major  performance  decrements. 
Conversely,  Williams  et  al.  (1997)  considered  the  low  rate  of  5  Hz  to  be  the  minimum  threshold 
for  continuous  speech  reading.  Another  important  aspect  of  speech  reading  is  the  potential  for 
audio  accompaniment,  which  can  be  skewed  at  lower  visual  FRs.  For  instance,  at  10  Hz,  the 
audio  presentation  should  lag  the  visual  presentation  by  at  least  +40  ms  (Knoche  et  al.,  2005). 

Observers  may  be  able  to  adapt  to  low  FRs,  however.  If  there  is  enough  opportunity  to  learn 
sign  language  gestures,  low  multimedia  rates  of  1  and  5  Hz  may  be  adequate  (Johnson  &  Caird, 
1996). 

Continuous  FR  presentation  in  a  purely  visual  signal  detection  task  demonstrated  an  apparent 
16-Hz  threshold  for  adequate  performance.  Increasing  FR  beyond  this  level  improved 
performance  only  mildly  (Whiting  et  al.,  1991). 

Another  approach  to  the  target  detection  environment  is  the  use  of  a  variable  FR  system.  The 
velocity-based  LOD  rendering  technique  is  one  such  approach  to  detennining  FR.  It  can  yield 
faster  graphics  rendering  times  when  high  LOD  is  not  necessary,  and  this  in  turn  can  improve 
visual  search  performance  (Parkhurst  &  Niebur,  2001). 

3.2  Perception  and  Psychophsics 

The  effects  of  slow  FRs  on  human  perception  have  been  examined  in  a  wide  variety  of 
environments.  For  example,  in  a  study  conducted  in  a  VE,  Piantanida,  Boman,  and  Gille  (as 
cited  in  Reddy,  1997)  found  that  participants’  depth  and  egomotion  perception  degraded  when 
FRs  dropped.  On  the  other  hand,  Van  Erp  and  Padmos  (2003)  assessed  the  effects  of  slow 
update  rates  on  the  viewing  system  for  indirect  driving  and  observed  that  an  update  rate  as  low  as 
3  Hz  did  not  appear  to  affect  participants’  speed  perception. 

Augmented  reality  (AR),  in  which  computer  graphics  are  overlaid  onto  the  physical  environment 
through  transparent  HMDs,  has  also  been  used  to  study  the  effects  of  slow  FRs.  Effective  AR 
depends,  in  part,  on  the  mechanisms  of  human  visual  perception,  such  as  depth  cues  and  motion 
detection.  Lai  and  Duh  (2004)  assessed  the  role  of  different  FRs  on  the  perception  of 
visualization  dynamic  infonnation  via  the  change  in  bar  graphs.  (The  bar  graphs  were  presented 
in  an  HMD.)  Users  can  visualize  changes  in  dynamic  information  displays  by  tracking  the 
motion  of  the  stimulus  (e.g.,  vertical  changes  bar  graph  height).  Users  can  also  anticipate  the 
future  motion  of  the  stimulus  change  as  a  function  of  the  implied  velocity  of  the  stimulus  (i.e., 
representational  momentum;  see  Freyd  &  Finke,  1985).  Thus,  the  display’s  ability  to  convey 
motion  can  impact  the  perception  of  dynamic  information  representation.  Participants  judged  the 
percentage  change  in  a  dynamic  3-D  bar  chart  (e.g.,  the  rate  at  which  the  height  of  a  bar  graph 
was  increasing).  Interestingly,  the  greatest  judgment  inaccuracies  were  made  at  the  fastest  FR, 
160  Hz.  It  was  found  that  participants  overestimated  the  percentage  at  lower  speeds  and 
underestimated  at  higher  speeds.  This  may  be  because  in  the  slower  speeds,  participants  use 
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cognitive  resistance  (see  Finke  &  Shyi,  1988)  in  later  stages  of  the  presentation  to  reduce  the 
degree  of  representational  momentum.  It  is  also  important  to  consider  the  typical  human 
cognitive  processing  speed  of  170  msec  (Card  et  ah,  1983)  since  changes  that  occur  below  this 
threshold  may  not  be  accurately  perceived.  This  may  explain  the  perfonnance  improvements  for 
the  longer  frame  times  (i.e.,  slower  FRs). 

Mastoropoulou  and  Chalmers  (2004)  hypothesized  that  music  could  be  used  as  a  distractor  to 
affect  the  user’s  perception  of  time  enough  to  de-emphasize  slower  rendering  times.  Participants 
with  computer  graphics  experience  were  tested.  Animated  clips  were  compared  at  different  FRs, 
and  each  compared  pair  had  a  difference  of  4  Hz  (16  versus  20  Hz  or  12  versus  16  Hz).  The  two 
types  of  music  that  accompanied  the  animations  were  “slow  tempo  and  relaxing”  and  “fast  tempo 
and  exciting”.  Frequency  analyses  showed  that  slow  tempo  music  decreased  the  perceived  scene 
velocity.  Contrary  to  the  hypothesis,  playing  fast  tempo  music  did  not  convince  observers  that 
the  clip  was  shorter  and  progressed  at  a  higher  temporal  rate.  The  lack  of  support  for  the 
hypothesis  could  be  because  a  difference  of  4  Hz  was  not  enough  for  participants  to  perceive  a 
velocity  difference  when  the  two  different  FRs  were  compared. 

Pfautz  (2002,  Experiment  E)  was  interested  in  the  impact  of  spatio-temporal  sampling  on  the 
perception  of  constant  velocity  motion.  There  was  an  interaction  between  FR  and  stimulus 
velocity,  so  that  higher  velocities  at  higher  FRs  (e.g.,  15  and  60  Hz  versus  5  and  7.5  Hz)  yielded 
more  accurate  response  distances.  In  a  second  related  experiment,  Pfautz  (2002,  Experiment  F) 
assessed  the  effects  of  viewpoint  manipulation  in  an  air  traffic  control  collision  task.  The  task 
involved  making  a  time-to-contact  judgment  for  two  moving  aircraft  on  a  screen.  The  higher 
FR,  20  Hz,  provided  greater  judgment  accuracy  than  7.5  Hz.  Allowing  participants  to  alter  their 
viewpoint  impaired  this  accuracy,  however. 

Reddy  (1997,  Experiment  1)  assessed  performance  in  a  virtual  heading  task  while  manipulating 
FR  at  either  2.3  Hz  (considered  to  be  excessively  low)  or  1 1.5  Hz  (considered  to  be  moderately 
acceptable).  In  the  heading  task,  the  subject  was  passively  navigated  through  the  VE  with  his  or 
her  direction  of  fixation  oriented  differently  from  the  direction  of  movement,  and  the  task  was  to 
determine  whether  he  or  she  was  moving  toward  the  left  or  right  of  the  fixation  point.  The 
higher  FR  of  1 1.5  Hz  had  clear  performance  benefits  in  terms  of  accuracy  and  task  completion 
time.  The  same  task  was  again  executed  in  a  second  experiment  (see  Reddy,  1997,  Experiment 
2).  Higher  FRs  were  manipulated  in  this  experiment,  this  time  with  6.7  Hz  and  14.2  Hz.  As  in 
the  first  experiment,  similar  trends  were  found,  so  that  accuracy  and  reaction  time  were  better  in 
the  higher  FR  condition.  More  precise  angle  discriminations  were  possible  at  this  higher  FR. 

3.2.1  Perception  and  Psychophsics  Conclusion 

Van  Erp  and  Padmos  (2003)  assessed  the  effects  of  slow  update  rates  of  the  viewing  system  for 
indirect  driving  and  observed  that  an  update  rate  as  low  as  3  Hz  did  not  appear  to  affect 
participants’  speed  perception.  Piantanida,  Boman,  and  Gille  (as  cited  in  Reddy,  1997),  on  the 


17 


other  hand,  found  that  participants’  depth  and  egomotion  perception  degraded  when  FRs 
dropped. 

The  perception  of  constant  velocity  motion  seems  to  be  best  served  by  higher  FRs  such  as  15,  20, 
and  60  Hz  (Pfautz,  2002,  Experiments  E  and  F).  However,  there  is  an  upper  limit  to  the  ideal 
FR.  Excessively  rapid  FRs  (e.g.,  160  Hz)  can  impair  the  apparent  rate  of  change  in  dynamic 
informational  displays  (Lai  &  Duh,  2004).  On  the  other  hand,  differences  of  4  Hz  at  modest  FRs 
(e.g.,  in  the  range  of  12  to  20  Hz)  may  be  undetected  by  viewers,  even  if  they  are  experienced 
with  computer  graphics  (Mastoropoulou  &  Chalmers,  2004). 

In  heading  performance,  there  is  a  strong  performance  benefit  when  the  FR  increases  to  10  to 
15  Hz;  however,  performance  improvements  taper  off  afterwards  at  higher  rates.  Thus,  it  seems 
that  15  Hz  might  be  considered  the  minimum  threshold  for  VEs.  FRs  below  10  Hz  will  yield 
sharp  performance  degradations,  in  terms  of  both  response  time  and  heading  accuracy.  Heading 
tasks  have  many  parallels  to  other  VE  applications,  such  as  driving  and  flight  simulators,  and  so 
the  results  can  extend  to  these  domains,  as  well  (Reddy,  1997). 


A  summary  of  these  research  findings  is  presented  graphically  in  figure  2  and  is  also  provided  in 
more  detail  in  appendix  B. 


Summary  of  Frame  Rate  Effects  on  Perceptual 
Performance 


♦  FR-Normal 
■  FR-Degrade 


Figure  2.  Summary  of  FR  effects  on  perceptual  performance. 
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4.  Behavioral  Effects 


Eye  movements  are  considered  important  in  our  understanding  of  visual  perception,  attention, 
and  cognitive  processes.  The  eye  will  focus  on  particularly  informative  areas  of  a  display  (Peiz, 
Canosa,  &  Babcock,  2000);  thus,  poor  visual  quality  may  fail  to  attract  eye  fixation.  Quality  of 
Service  (QoS)  refers  to  the  goodness  or  degree  to  which  the  multimedia  presentation  is 
considered  tolerable  in  terms  of  viewing  experience  and  is  partially  determined  by  FR,  which,  in 
turn,  is  attributable  to  bandwidth  limitation  implications. 

Gulliver  and  Ghinea  (2004a)  investigated  changes  in  eye  movements  as  influenced  by  low  FRs. 

It  was  hypothesized  that  jerkier  and  disjointed  video  presentations  from  the  low  FRs  would  be 
more  perceptually  intolerable  and  annoying,  thereby  causing  more  variability  in  eye  paths. 
Interestingly,  no  correlation  was  found  between  FR  and  eye  path  movements.  Thus,  Gulliver 
and  Ghinea  (2004a)  suggest  that  lower  FRs  (e.g.,  5  Hz)  do  not  influence  the  flow  of 
informational  assimilation. 

Individuals  are  able  to  conduct  video-mediated  conversations  with  the  use  of  hand  motions  at 
FRs  as  low  as  1  Hz  (Chen,  2002)  and  one  frame  change  every  5  seconds,  although  subjective 
measures  indicated  that  participants  thought  this  extremely  low  FR  was  ineffective  (Chen,  2003). 
Thus,  despite  noticeably  low  temporal  quality,  these  types  of  visual  cues  can  retain  their 
effectiveness. 

Extremely  slow  FRs  and  their  associated  visual  lag  may  cause  cue  conflict  (i.e.,  discrepancy 
between  visual  and  vestibular  systems),  which  has  been  shown  to  induce  cyber/simulator 
sickness  (Kolasinski,  1995;  Stanney,  Mourant,  &  Kennedy,  1998). 


5.  Subjective  Assessment 


Varying  FRs  may  influence  the  subjective  ratings  of  video  quality  and  viewing  enjoyment.  This 
may  be  of  importance  to  service  providers  who  need  to  know  the  minimum  level  of  quality  that 
their  customers  find  acceptable,  as  well  as  viewers  who  need  to  be  able  to  recognize  the  subject 
matter  and  observe  smoother  movements  (McCarthy,  Sasse,  &  Miras,  2004). 

QoS  involves  spatial  (intraframe)  characteristics  (e.g.,  picture  resolution,  color  depth),  temporal 
(interframe)  characteristics  (e.g.,  FR),  as  well  as  a  spatiotemporal  integration  (e.g.,  as  auditory 
effects  that  are  in  synchrony  with  the  visual  cues)  (Apteker,  Fisher,  Kisimov,  &  Neishlos,  1995). 
It  is  related  to  the  image  quality  or  FR  trade-off  in  bandwidth  allocation,  in  which  FR  can  be 
reduced  in  order  to  improve  the  spatial  aspects  of  the  video. 
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“Watchability”  is  a  characteristic  of  video  which  includes  the  viewer’s  acceptance  of  the  video’s 
audio  signals,  the  continuity  of  visual  messages,  lip  movement-speech  synchronization,  and  the 
relationship  between  the  audio  and  visual  message  components  (Apteker  et  ah,  1995).  Lowering 
FRs  has  been  shown  to  reduce  the  watchability  of  video  clips,  so  that  15  Hz  might  be  just 
acceptable,  10  Hz  is  much  less  acceptable,  and  5  Hz  is  very  unacceptable  (Apteker  et  ah).  High 
action  videos  (e.g.,  sports  clips)  on  a  desktop  display  with  a  resolution  of  352  x  288  pixels  have 
yielded  similar  satisfaction  ratings  across  the  FRs  of  25,  15,  and  5  Hz  (Ghinea  &  Thomas,  1998) 
and  have  been  considered  acceptable  80%  of  the  time  when  FRs  were  only  6  Hz  (McCarthy  et 
ah,  2004).  When  videos  were  viewed  on  a  palmtop  display  (sized  176  x  144),  low  FR  was  found 
to  be  less  acceptable  than  on  a  desktop  display,  so  that  the  critical  lowest  acceptable  FR  value 
was  12  Hz,  and  6  Hz  was  considered  acceptable  only  50%  of  the  time. 

Gulliver  and  Ghinea  (2004b,  2004c,  2004d)  found  that  degradation  of  perceived  video  quality 
and  satisfaction  was  not  noticeable  when  FR  was  reduced  from  25  Hz  to  15  Hz,  but  it  was 
noticeable  when  the  FR  was  reduced  to  5  Hz.  Thus,  presentations  made  at  15  Hz  and  25  Hz 
appear  perceptually  similar  to  the  participants.  An  FR  reduction  from  30  to  1  frame  changes 
every  5  seconds  had  minimal  effect  on  engagement  and  enjoyment  in  a  video-mediated 
conversation  (Chen,  2003).  Other  studies  have  found  that  reductions  in  FR  (i.e.,  from  25  to  15, 
and  then  to  5  Hz)  often  do  not  impair  viewing  enjoyment;  however,  there  is  a  reduction  in 
perceived  quality  (Gulliver,  Serif,  &  Ghinea,  2004). 

The  relationship  between  FR  and  perceived  quality  may,  however,  be  context  dependent.  Masry 
and  Hemami  (2001)  found  that  perceived  quality  was  higher  for  lower  FRs  (such  as  10  and 
15  Hz)  in  higher  action  sequences,  but  it  was  found  unacceptable  in  clips  involving  less  motion; 
this  indicates  a  possible  preference  specific  to  motion  types.  In  a  similar  vein,  video  clip  subject 
matter  may  result  in  different  levels  of  perceptual  quality  and  user  satisfaction,  independent  of 
FR  (Gulliver  &  Ghinea  (2004b).  For  example,  across  all  FRs  (5,  15,  and  25  Hz),  popular  music 
videos  had  higher  subjective  quality  and  satisfaction  ratings  than  a  chorus  video. 

Quality  of  perception  (QoP)  is  the  perceptual  experience  of  the  user,  including  the  enjoyment  of 
watching  the  video,  as  well  as  the  ability  to  analyze,  synthesize,  and  assimilate  information 
presented  (see  Ghinea  &  Thomas,  1998;  Ghinea  &  Chen,  2003;  Gulliver  &  Ghinea,  2004a, 
2004b,  2004c,  2004d).  Thus,  QoP  involves  the  viewer’s  information  processing.  Multimedia 
can  often  be  greatly  degraded  before  perception,  information  assimilation,  and  learning  are 
impaired.  Ghinea  and  Thomas  (2000)  demonstrated  that  viewers  could  integrate  visual  details  to 
learn  and  then  correctly  answer  questions  about  the  content  of  videos,  even  as  FR  was  reduced 
from  25  to  5  Hz.  Often,  lower  FRs  (5  Hz)  resulted  in  more  correct  answers  about  what  had  been 
viewed,  possibly  because  there  was  more  time  to  view  frames  before  they  changed  (at  25  Hz, 
each  frame  is  visible  for  40  ms,  but  at  5  Hz,  a  frame  is  visible  for  200  ms).  The  robustness  of 
information  assimilation  despite  low  FR  has  been  documented  in  other  studies  (see  Ghinea  & 
Thomas,  1998;  Gulliver  &  Ghinea,  2000;  Ghinea  &  Chen,  2003;  Gulliver  &  Ghinea,  2004a, 
2004b,  2004c,  2004d;  Gulliver,  Serif  et  al„  2004). 
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According  to  Ghinea  and  Chen  (2003),  cognitive  style,  such  as  field  dependence  (i.e.,  degree  to 
which  a  person’s  perception  of  information  is  influenced  by  the  contextual  field)  and  field 
independence,  were  not  moderators  in  the  relationship  between  FR  and  QoP.  Furthermore,  eye 
path  movement,  which  is  associated  with  attentional  and  cognitive  processes,  did  not  differ 
across  FRs  of  5,  15,  or  25  Hz  (Gulliver  &  Ghinea,  2000d). 


Van  Breda,  Jansen,  and  Veltman  (2005)  reported  that  UAV  operators  assessed  their  effort  in 
viewing  the  simulated  sensor  feed  as  higher  when  the  frame  rate  for  the  camera  image  was  low 
(i.e.,  3  Hz)  compared  to  normal  frame  rates.  Wilson  and  Sasse  (2000)  have  shown  that  low  FRs 
can  cause  viewers  physiological  strain,  despite  the  fact  that  most  of  them  did  not  report  noticing 
a  difference  between  videos  presented  at  5  and  25  Hz.  Exposure  to  the  5-Hz  condition  resulted 
in  greater  experiences  of  stress,  characterized  by  increased  galvanic  skin  response,  increased 
heart  rate,  and  decreased  blood  volume  pulse.  The  results  of  this  study  suggest  that  it  may  not  be 
sufficient  to  measure  subject  reactions  alone,  since  physiological  effects  can  also  occur.  This 
partially  corroborates  the  findings  of  Meehan  et  al.  (2002),  in  which  10  Hz  caused  large 
increases  in  heart  rate  compared  to  higher  FRs. 


A  summary  of  these  research  findings  is  presented  graphically  in  figure  3  and  is  also  provided  in 
more  detail  in  appendix  C.  Figures  1  through  3  are  summarized  graphically  in  figure  4. 


Summary  of  Frame  Rate  Effects  on  Subjective 
Assessment 
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Figure  3.  Summary  of  FR  effects  on  subjective  assessment. 
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Figure  4.  Summary  of  effects  of  slow  FR  (figures  1  through  3).  (Note:  The  x-axis  denotes  the 

number  that  the  study  was  designated.  The  list  of  studies  with  their  corresponding  numbers 
is  provided  in  appendices  A  through  C.  Pink  squares  signify  the  FR  level  where  human 
performance  starts  to  significantly  degrade.  Blue  diamonds  signify  the  lowest  FR  levels 
where  participants  perfonn  normally.) 


6.  Conclusions 


Robotic  systems  of  various  sizes  and  capabilities  will  play  an  integral  role  in  the  U.S.  Army’s 
FCS  program  (Barnes,  Cosenzo,  Mitchell,  &  Chen,  2005).  According  to  Mait  and  Grossman 
(2002), 


...although  the  operational  concept  for  FCS  requires  UGVs  to  sense  the 
battlefield  and  react  on  their  own  with  minimal  human  interaction,  current 
technology  can  best  be  described  as  remote  controlled  or  teleoperated. 
Semiautonomous  operation,  suitable  for  sensing  and  indirect  fire  functions,  will 
not  be  available  until  2010,  and  fully  autonomous  systems  (necessary  for  direct 
fire,  battle  damage  assessment,  and  reconnaissance,  surveillance,  targeting,  and 
acquisitions)  will  not  be  available  until  2015  or  later,  (http://www.ndu.edu/ 
inss/DefHor/DH  1 3/DH 1 3  .htm) 

Effectiveness  of  employing  these  robotic  assets,  especially  if  teleoperation  will  be  involved,  will 
in  large  part  be  detennined  by  the  quality  of  the  video  feeds  of  the  remote  environment,  which 
will  be  the  main  source  of  information  for  the  robotic  operator  to  perform  his  or  her  tasks  (e.g., 
target  detection,  tele-manipulation).  Since  bandwidth  constraints  may  be  an  issue  in  the  future 
battlefield  (Mait  &  Grossman,  2002)  and  high  FR  streaming  video  may  not  be  available  at  all 
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times,  it  is  important  to  examine  human  performance  issues  that  may  be  associated  with  low 
FRs. 


In  this  report,  we  reviewed  more  than  50  studies  and  summarized  them  in  the  areas  of 
psychomotor  perfonnance,  perceptual  performance,  behavioral  effects,  and  subjective 
perception.  Generally,  psychomotor  performance  improves  at  higher  FRs  and  lower  standard 
deviations  of  FR.  In  placement  tasks,  4  Hz  may  be  too  low  for  VE  perfonnance,  and  10  Hz  may 
disrupt  human  balance.  Experimental  results  have  suggested  a  minimum  threshold  of  17.5  Hz 
for  successful  placement  performance  (Watson  et  ah,  1997).  Tracing  performance  may  also 
require  more  than  10  Hz. 

For  teleoperation,  higher  FRs  such  as  16  Hz  are  suggested  to  aid  in  navigation  and  target 
tracking.  A  rate  of  approximately  10  Hz  may  suffice  for  HMD-facilitated  tracking.  This  lower 
required  FR  will  be  useful  for  releasing  bandwidth  for  other  uses,  such  as  rendering  more 
graphically  detailed  images. 

Speech  reading  is  an  often-studied  fonn  of  target  recognition.  Generally,  13  Hz  may  be  too  low 
for  speech  reading  from  video  transmission;  rather,  at  least  15  Hz  may  be  necessary.  However, 
with  appropriate  learning  opportunity,  rates  around  5  Hz  (and  possibly  less)  may  be  sufficient. 

As  with  video  speech  reading,  target  detection  in  a  visual  radiological  task  may  be  significantly 
more  successful  at  16  Hz.  Perceptual  VE  task  performance  seems  to  be  best  served  by  FRs 
above  15  Hz.  This  includes  perceiving  motion  of  a  constant  velocity  and  heading  performance. 

Individuals  seem  to  be  able  to  gather  content-based  information  about  videos  that  are  viewed  at 
low  FRs  such  as  5  Hz.  While  this  FR  is  very  low,  it  may  benefit  information  assimilation 
because  each  frame  remains  on  the  screen  for  a  relatively  longer  duration  compared  to  frames 
that  are  presented  at  higher  rates.  This  provides  viewers  with  more  time  to  observe  each  frame. 
Eye  movement  studies  support  these  findings;  the  lack  of  effect  for  FR  upon  eye  movements 
indicates  that  there  are  not  significant  changes  in  attentional  and  cognitive  processes. 

In  terms  of  watchability  and  satisfaction,  FRs  of  5  Hz  can  appear  “unacceptable”  and  of  lower 
visual  quality.  Still,  they  are  often  considered  to  be  enjoyable  to  watch  despite  the  temporal 
distortion.  However,  FR  reductions  from  25  to  15  Hz  are  often  perceptually  similar  in 
appearance  to  viewers. 

Also,  FRs  at  10  Hz  and  below  have  been  shown  to  cause  stress  in  tenns  of  physiological 
reactions  in  addition  to  general  perfonnance  decrements. 

Overall,  there  seems  to  be  strong  support  for  a  threshold  of  around  15  Hz  for  many  tasks, 
including  those  that  are  psychomotor  and  perceptual  in  nature.  Only  Lion  (1993)  and  Watson  et 
al.  (1998,  Experiment  2)  (of  the  56  studies  reviewed)  showed  significant  performance 
degradation  associated  with  FRs  higher  than  15  Hz.  Less  impressive  yet  acceptable  performance 
may  be  accomplished  at  around  10  Hz  for  many  tasks.  However,  it  is  important  to  consider  that 
individuals  may  be  more  susceptible  to  stress  symptoms  at  FRs  at  and  below  10  Hz.  Information 


23 


assimilation  seems  to  be  far  more  impervious  to  the  detrimental  effects  of  FR,  since  participants 
can  observe  and  retain  much  infonnation  from  clips  presented  at  rates  as  low  as  5  Hz.  Subjective 
reactions  to  quality  and  watchability  of  videos  seem  to  support  rates  of  5  Hz,  although  videos 
presented  at  15  Hz  and  above  are  generally  more  widely  preferred. 

These  generalizations  regarding  superior  and  acceptable  FRs  may  also  be  subject  to  the  effects  of 
several  moderators.  The  majority  of  the  studies  reviewed  have  involved  other  independent 
variables  and  equipment  characteristics  that  could  be  contributing  to  or  detracting  from  the 
proposed  effects  of  various  FRs. 

The  use  of  variable  FR  manipulation  has  been  recommended  over  constant  FRs  because  of  its 
proposed  accuracy  advantage  (Watson  et  ah,  1997).  There  are  many  different  ways  to  determine 
the  standard  deviations  in  laboratories;  however,  some  techniques  may  be  more  accurate  in  terms 
of  how  FR  truly  varies  in  real-world  systems  (e.g.,  standard  deviations  based  on  percentages  of 
mean  FRs). 

Stereoscopic  displays  will  likely  enable  a  performance  advantage  over  monoscopic  displays,  and 
when  possible,  they  should  be  used  to  assist  operators  in  compensating  for  lower  FR 
presentations.  If  displays  are  stereoscopic,  FRs  as  low  as  7  Hz  may  be  adequate,  but  monoscopic 
displays  may  only  be  useful  for  FRs  as  low  as  14  Hz  (Richard  et  ah,  1996). 

User  experience  may  also  be  an  important  moderator.  Individuals  with  HMD  use  experience 
were  able  to  tolerate  FRs  as  low  as  10  Hz  in  tracking,  whereas  inexperienced  participants  needed 
30  Hz  (Liu  et  ah,  1993).  This  difference  in  tolerable  FR  is  quite  large,  and  so  it  is  clear  that 
demographics  should  be  measured  and  controlled  when  temporal  distortion  is  manipulated. 
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Appendix  A.  Summary  of  FR  Effects  on  Psychomotor  Performance 


Study 

No. 

Author(s) 
and  Date 

FR  Manipulation 

Task  Type 

Measures 

Findings 

1 

Arthur  et  al. 
(1993) 

10, 15,  30  Hz 

3-D  tree¬ 
tracing 

Accuracy 

Speed 

Highest  response  time  at  lowest  FR  (10  Hz 
condition). 

2 

Bryson  (1993, 
Experiment  1) 

2,  3,  4,  6,  10,  20, 

30,  60  Hz 

Tracking 

Accuracy 

Longer  frame  times  (i.e.,  lower  FRs)  at  higher 
frequencies  of  target  motion  produced  the 
greatest  tracking  error. 

2 

Bryson  (1993, 
Experiment  2) 

2,  3,  4,  6,  10,  20, 

30,  60  Hz 

Placement 

Accuracy 

Speed 

Performance  worsened  by  increased  lag  and 
lower  FR,  especially  frame  times  longer  than 
0.25  sec.  Experienced  participants  less 
impaired  by  low  FR. 

3 

Cai  (2004) 

Range  from  0.5,  1, 
to  17  Hz 

Tracking  a 
moving  human 
figure 

Accuracy 

Minimal  satisfactory  FR  was  1  Hz  for  this 
type  of  task  in  normal  viewing  conditions. 

4 

Day  (1999) 

1,2,  3,4,  and  25  Hz 

Tracking 

Accuracy 

Speed 

25  Hz  is  ideal;  overall  performance  is  worse 
for  FRs  of  1  to  4  Hz. 

5 

Ellis  et  al. 

(1999) 

6,  12,  and  20  Hz 

Tracking 

Accuracy 
Perceived 
control  & 
stability 

Decreased  FR  significantly  increased  rms 
error,  increased  adapted  Cooper-Harper  scale 
scores,  and  decreased  stability. 

6 

French  et  al. 
(2003) 

2,  4,  8,  and  16  Hz 

Tracking 

Accuracy 

Speed 

16  Hz  is  ideal  for  UGV  teleoperation.  To  get 
effective  task  performance,  at  least  8  Hz 
should  be  used. 

7 

Lampton  et  al. 
(1994) 

5  Hz 

3-D  Tracking 

Accuracy 

Participants  could  only  keep  the  cursor  on  the 
moving  target  less  than  9%  of  the  time. 

8 

Lion  (1993) 

22  and  33  Hz 

Tracking 

Accuracy 

33  Hz  FR  yielded  better  performance  than  22 

Hz  FR. 

9 

Liuet  al.  (1993) 

0.1,0.25,0.5,  1, 

1.5,  2,  3,  5,  10  &  30 
Hz 

Tracking- 

experienced 

participants 

Accuracy 

Speed 

Suggest  a  minimum  update  rate  of  2  Hz  for 
experienced  VE  users. 

10 

Liuet  al.  (1993) 

0.1,0.25,0.5,  1, 

1.5,  2,  3,5,  10  &  30 
Hz 

Tracking- 

inexperienced 

participants 

Accuracy 

Speed 

Suggest  a  minimum  update  rate  of  10  Hz  for 
inexperienced  users. 

11 

Liuet  al.  (1993) 

0.25,  1,  1.5,2,  3,  10 
&  30  Hz 

Placement 

Accuracy 

Speed 

Suggest  a  minimum  update  rate  of  2  Hz  for 
experienced  VE  users  and  10  Hz  for 
inexperienced  users. 

12 

McGovern 

(1991) 

7.5,  30  Hz 

Tracking 

Vehicle 

Control 

Low  FR  did  not  significantly  affect  driving 
performance  on  a  cone-marked  route. 

13 

Massimino  & 
Sheridan  (1994) 

3,  5,  30  Hz 

Placement 

Accuracy 

Speed 

Task  completion  time  at  3  Hz  was 
significantly  slower  than  5  and  30  Hz.  While 

FR  of  30  Hz  is  best,  reduced  FRs  are  viable, 
especially  if  force  feedback  is  provided. 

14 

Richard  et  al. 
(1996) 

1,2,3,  7,  14,  28  Hz 

Tracking  & 
Grasping  - 
Mono 
condition 

Accuracy 

Speed 

Within  the  mono  condition,  completion  time 
was  fastest  and  stable  from  28  Hz  down  to  14 

Hz. 

15 

Richard  et  al. 
(1996) 

1,2,  3,  7,  14,  28  Hz 

Tracking  & 
Grasping  - 
Stereo 
condition 

Accuracy 

Speed 

Within  the  stereo  condition,  it  was  fastest  and 
remained  stable  from  28  Hz  down  to  7  Hz. 

16 

Van  Erp  & 
Padmos  (2003) 

3,  5, 10,  30  Hz 

Tracking 

Vehicle 

Lateral 

Control 

Lateral  control  of  vehicle  movement  was 
degraded  when  FR  dropped  below  5  Hz. 

17 

Ware  & 

Balakrishnan 

(1994) 

0.666,  1,2,  3,  5,  10, 
15,  60  Hz 

Target 

acquisition  in 
a  3-D 

environment 

Speed 

10  Hz  is  approximately  the  lowest  threshold 
for  adequate  target  acquisition  performance. 
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18 

Watson  et  al. 
(1997) 

10  and  20  Hz 

Placement 

Speed 

Accuracy 

Performance  significantly  improves  when 
average  FR  increases  to  20  Hz.  Average 
number  of  grab  attempts  and  percentage  of 
correct  attempts  were  significantly  lower  at  20 
Hz. 

19 

Watson  et  al. 
(1998, 

Experiment  1) 

9,  13,  and  17  Hz 

Placement 

Grasping 

Speed 

Accuracy 

Grasping  performance  may  be  compromised 
only  at  lower  mean  system  responsiveness 
(MSR)  FRs  (such  as  9  and  13  Hz  MSRs.  A 

2Hz  SDSR  may  be  the  lowest  threshold  at 
which  grasp  time  performance  is  affected. 

Lower  standard  deviations  (e.g.,  0.5  Hz)  may 
not  impact  grasping  time. 

20 

Watson  et  al. 
(1998, 

Experiment  2) 

17,  25,  and  33  Hz 

Placement 

Grasping 

Speed 

Accuracy 

Better  placement  perfonnance  at  higher  MSRs 
and  lower  SDSRs.  There  may  be  a  grasping 
performance  threshold  at  17  Hz. 

21 

Watson  et  al. 
(1998, 

Experiment  3) 

mean  FRs  17,  33, 
and  41  Hz,  each 
with  SDs  of  0.50, 
3.78,  and  7.56  Hz 

Placement 

Grasping 

Speed 

Accuracy 

Increase  in  grasp  time  only  when  the  FR  was 
at  the  lowest  (17  Hz)  and  SDSR  was  highest 
(7.56  Hz).  Placement  time  was  longer  when 

FR  was  17  Hz. 

32 


Appendix  B.  Summary  of  FR  Effects  on  Perceptual  Performance 


Study 

No. 

Author(s) 
and  Date 

FR  Manipulation 

Task  Type 

Measures 

Findings 

22 

Chen  et  al. 

(2005) 

5  and  30  Hz 

Target 

detection 

Accuracy 

No  differences  between  the  FRs. 

23 

French  et  al. 
(2003) 

2,  4,  8,  and  16  Hz 

Target 

identification 

Accuracy 

No  differences  among  the  FRs. 

24 

Frowein  et  al. 
(1991) 

5,  6,  7.5, 10, 15,  and 
30  Hz 

Target 
recognition 
&  Info 
processing 

Verbal 

information 

recall 

FRs  up  to  15  Hz  can  improve  speech  reading 
perception.  FRs  higher  than  15  Hz  do  not 
further  improve  performance. 

25 

Ghinea  & 

Thomas  (2000) 

5,  15,  25  Hz 

Information 

processing 

Information 

assimilation 

Information  assimilation  can  be  successful  at 

5  Hz,  and  this  low  FR  may  provide  the  viewer 
with  more  time  to  learn  the  content  of  the 
frame. 

26 

Gulliver  & 

Ghinea  (2004a) 

5,  15,  and  25  Hz 

Information 

processing 

Eye  path 
movement 

No  correlation  was  found  between  FR  and  eye 
path  movements.  In  this  setting,  5  Hz  was 
equally  effective  as  25  Hz. 

27 

Gulliver  & 

Ghinea  (2004,  b, 
c,  &  d) 

5,  15,  and  25  Hz 

Information 

processing 

Information 

assimilation 

Eye  path 
movement 

Low  FR  of  5  Hz  has  no  effect  on  information 
assimilation.  Eye  path  movement,  which  is 
associated  with  attentional  and  cognitive 
processes,  did  not  differ  across  FRs  of  5,  15, 
or  25  Hz. 
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Gulliver,  Serif, 

&  Ghinea, 

(2004) 

5,  15,  25  Hz 

Information 

processing 

Information 

assimilation 

Low  FR  of  5  Hz  has  no  effect  on  information 
assimilation. 
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Johnson  & 

Caird  (1996) 

1,5,  15,  or  30  Hz 

Target 

recognition 

Information 

processing 

Learning 

1  and  5  Hz  may  be  adequate  for  learning  these 
types  of  gestures  in  multimedia,  if  there  is 
enough  learning  opportunity. 
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Knoche  et  al. 
(2005) 

10,  15,30  Hz 

Target 

recognition 

Accuracy 

In  a  “noisy”  environment,  video  at  10  Hz  will 
be  effective  if  audio  lags  visual  by  +120  ms  to 
+170  ms.  If  the  audio  lags  by  more  than  170 
ms,  a  full  FR  of  30  Hz  should  be  reduced  to 

10  Hz. 
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Lai  &  Dull 
(2004) 

10,  40,  80,  120,  160 
Hz. 

Judgment  of 
visual 
percentage 
change 

Accuracy 

Greatest  inaccuracies  were  made  at  the  fastest 
FR,  160  Hz  (excessively  rapid).  In  AR, 
participants  underestimated  the  percentage 
increase,  except  for  in  the  10-Hz  condition. 

32 

Mastoropoulou 
&  Chalmers 
(2004) 

16  vs.  20  Hz 

or 

12  vs.  16  Hz 

Viewing 

animated 

clips 

Temporal 

perception 

A  difference  of  4  Hz  was  not  enough  for 
participants  with  computer  experience  to 
perceive  a  velocity  difference  between  the  two 
FRs  (12  vs.  16  Hz  and  16  vs.  20  Hz). 

Parkhurst  & 
Niebur  (2001) 

Constant  at  6  Hz,  or 
varied  (median  FRs 
ranged  from  9  to 

31.5  Hz) 

Target 

recognition 

Accuracy 

Reaction  time 

FRs  and  frame 

rendering 

times 

(computer 

performance) 

Reaction  times  increased  with  decreasing 

LOD  in  the  constant  FR  condition,  but  in  the 
variable  FR  condition,  reaction  times 
decreased  when  LOD  decreased. 
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Pfautz  (2002, 
Experiment  E) 

5,  7.5, 15,  60  Hz 

Time-to- 

contact 

judgment 

Accuracy 

No  significant  main  effect  for  FR.  Interaction 
between  FR  and  velocity;  higher  velocities  at 
higher  FRs  yielded  more  accurate  response 
distances. 

34 

Pfautz  (2002, 
Experiment  F) 

7.5  Hz  vs.  20  Hz 

Time-to- 

contact 

judgment 

Accuracy 

20  Hz  FR  provided  greater  judgment 
accuracy. 

33 


35 

Reddy  (1997, 
Experiment  1) 

2.3  Hz  and  11.5  Hz 

Directional 

(orientation) 

judgment 

Accuracy 
Reaction  time 
in  heading 
task 

1 1 .5  Hz  had  significantly  better  performance 
than  2.3  Hz  condition. 

36 

Reddy  (1997, 
Experiment  2) 

6.7  Hz  and  14.2  Hz 

Directional 

(orientation) 

judgment 

Accuracy 
Reaction  time 
in  heading 
task 

1 5  Hz  might  be  considered  the  minimum  for 
VEs,  and  higher  FRs  will  continue  to  show 
better  performance,  but  at  a  reduced  rate.  FRs 
below  10  Hz  will  yield  sharp  performance 
degradations,  in  terms  of  both  response  time 
and  heading  accuracy. 
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Van  Erp  & 
Padmos  (2003) 

3,  5,  10,  30  Hz 

Perception- 
Speed  & 
Distance 
Estimation 

Accuracy 

No  effects  were  observed. 

38 

Vitkovitch  & 
Barber  (1994) 

12.5, 16.7,  25  Hz 

Target 
recognition- 
lip  reading 

Accuracy 

Performance  improved  when  FR  was 
increased.  16.7  Hz  may  be  adequate  for 
recognition  tasks  of  this  nature. 

38 

Vitkovitch  & 
Barber  (1996, 
Experiment  1) 

8.3, 12.5, 16.7,  25 

Hz 

Target 
recognition- 
lip  reading 

Accuracy 

Accuracy  significantly  improved  from  53%  to 
69%  correct  when  FRs  were  increased  from 

8.3  Hz  to  25  Hz.  The  change  from  8.3  Hz  to 
16.7  Hz  was  also  significant  (52  %  to  61% 
accuracy). 
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Vitkovitch  & 
Barber  (1996, 
Experiment  2) 

8.3,  12.5,  and  25  Hz 

Target 
recognition- 
lip  reading 

Accuracy 

FRs  higher  than  8.3  Hz  are  necessary  for  lip 
reading,  with  16.7  Hz  being  adequate. 
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Whiting  et  al. 
(1991) 

0,  2,  4,  8,  16,  and  32 
Hz 

Target 

detection 

Accuracy 

Correct  detection  decreased  as  FR  decreased. 

16  Hz  may  be  the  lowest  acceptable  FR. 
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Williams  et  al. 
(1997) 

2,  5,  10,  15,  and  30 

Hz 

Target 

recognition- 

speech 

reading 

Accuracy 

5  Hz  is  considered  to  be  the  minimum 
requirement  for  continuous  speech  reading 
presentation. 
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Appendix  C.  Summary  of  FR  Effects  on  Subjective  Assessment 


Note:  FR  values  in  Red  signify  the  FR  levels  where  human  performance  starts  to  significantly 
degrade.  FR  values  in  Blue  signify  the  lowest  FR  levels  where  participants  perfonn  normally. 


Study 

No. 

Author(s) 
and  Date 

FR  Manipulation 

Task  Type 

Measures 

Findings 

42 

Apeteker  et  al. 
(1995) 

5,  10,  15  Hz 

Viewing  video 
clips 

Watchability 

QoS 

15  Hz  considered  “just  acceptable”  for 
watchability. 

43 

Chen  (2003) 

0.2  &  15  Hz, , 
variable  (motion- 
sensitive) 

Conversational 

behavior 

Perceived 

engagement/ 

enjoyment 

FR  of  1  new  image  per  5  seconds  is 
considered  acceptable  for  engagement  and 
enjoyment  in  video-mediated  conversation, 
but  “ineffective”  for  perceiving  hand 
movements;  15  Hz  considered  to  be  useful. 

44 

French  et  al. 
(2003) 

2,  4,  8,  and  16  Hz 

Teleoperating 
ground  robots 

Perceived 

workload 

FR  of  2  Hz  is  associated  with  significantly 
greater  workload 

45 

Ghinea  &  Chen 
(2003) 

5,  15,  25  Hz 

Information 

assimilation 

QoP 

Information  assimilation  and  enjoyment  can 
occur  for  films  presented  at  5  Hz. 

46 

Ghinea  & 

Thomas  (1998) 

5,  15,  25  Hz 

Information 

assimilation 

QoP 

5  Hz  can  impair  satisfaction;  5  Hz  may  be 
acceptable  in  terms  of  watchability  for  high 
action  and  low  resolution  video. 

47 

Gulliver  & 

Ghinea  (2004,  b, 
c,  &  d) 

5,  15,  and  25  Hz 

Information 

assimilation 

QoP 

Low  FR  of  5  Hz  has  no  effect  on  information 
assimilation.  Level  of  enjoyment  and  level  of 
quality  are  impaired  when  reducing  FR  from 

15  to  5  Hz.  Thus,  15  Hz  may  be  the  critical 
value  for  QoP. 

48 

Gulliver,  Serif, 

&  Ghinea, 

(2004) 

5,  15,  25  Hz 

Information 

assimilation 

QoP 

Low  FR  of  5  Hz  has  no  effect  on  enjoyment 

49 

Masry  & 

Hcmami 

(2001) 

10,  30  Hz 

Viewing 
motion  clips 

Perceived 
quality 
(lower 
motion  clips) 

Perceived  quality  can  be  content  dependent. 

10  Hz  is  acceptable  for  higher  action 
sequences,  but  not  lower  motion  clips. 

50 

Masry  & 

Hcmami 

(2001) 

10,  30  Hz 

Viewing 
motion  clips 

Perceived 

quality 

(higher 

action 

sequences) 

Perceived  quality  can  be  content  dependent. 

10  Hz  is  acceptable  for  higher  action 
sequences,  but  not  lower  motion  clips. 

51 

McCarthy  et  al. 
(2004, 

Experiment  1) 

6,  10,  12,  15,  18, 

20,  24  Hz 

Viewing  video 
on  desktop 
display 

Visual 
acceptability 
(subjective) 
Proportion  of 
eye 

movements 

At  6  Hz,  desktop  quality  was  acceptable  80% 
of  the  time. 

52 

McCarthy  et  al. 
(2004, 

Experiment  2) 

6,  10,  12,  15,  18, 

20,  24  Hz 

Viewing  video 
on  palmtop 
display 

Visual 

acceptability 

(subjective) 

Critical  value  for  FR  in  the  palmtop  is  12  Hz, 
but  6  Hz  is  acceptable  50%  of  the  time. 

53 

Meehan  et  al. 
(2002) 

10,  15,  20,  and  30 

Hz 

Placement  task 

Physiological 

Subjective 

presence 

Lower  FRs  can  impair  performance  and 
feelings  of  presence;  10  Hz  causes  subjective 
discomfort.  The  increase  in  heart  rate  was 
greater  at  10  Hz  than  at  15  Hz,  indicating 
higher  stress  at  10  Hz. 

54 

Van  Breda  et  al. 
(2005) 

3,  10,  and  30  Hz 

Viewing  2-D 
vs.  3-D  maps 

Subjective 
effort  rating 

Greater  perceived  effort  associated  with  3  Hz; 
positive  effect  of  3-D  map  was  largest  when 

FR  was  lowest 

55 

Wilson  &  Sasse 
(2000) 

5  and  25  Hz 

Viewing  video 

Subjective 
quality  rating 

5  Hz  is  acceptable  for  subjective  quality 
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56 

Wilson  &  Sasse 
(2000) 

5  and  25  Hz 

Viewing  video 

Physiological 

measure 

(stress) 

5  Hz  caused  greater  stress  (as  evidenced  by 
physiological  measures) 

42 

Apeteker  et  al. 
(1995) 

5,  10,  15  Hz 

Viewing  video 
clips 

Watchability 

QoS 

15  Hz  considered  “just  acceptable”  for 
watchability. 

43 

Chen  (2003) 

0.2  &  15  Hz, , 
variable  (motion- 
sensitive) 

Conversational 

behavior 

Perceived 

engagement/ 

enjoyment 

FR  of  1  new  image  per  5  seconds  is 
considered  acceptable  for  engagement  and 
enjoyment  in  video-mediated  conversation, 
but  “ineffective”  for  perceiving  hand 
movements;  15  Hz  considered  to  be  useful. 

44 

French  et  al. 
(2003) 

2,  4,  8,  and  16  Hz 

Teleoperating 
ground  robots 

Perceived 

workload 

FR  of  2  Hz  is  associated  with  significantly 
greater  workload 

45 

Ghinea  &  Chen 
(2003) 

5,  15,  25  Hz 

Information 

assimilation 

QoP 

Information  assimilation  and  enjoyment  can 
occur  for  films  presented  at  5  Hz. 

46 

Ghinea  & 

Thomas  (1998) 

5,  15,  25  Hz 

Information 

assimilation 

QoP 

5  Hz  can  impair  satisfaction;  5  Hz  may  be 
acceptable  in  terms  of  watchability  for  high 
action  and  low  resolution  video. 

47 

Gulliver  & 

Ghinea  (2004,  b, 
c,  &  d) 

5,  15,  and  25  Hz 

Information 

assimilation 

QoP 

Low  FR  of  5  Hz  has  no  effect  on  information 
assimilation.  Level  of  enjoyment  and  level  of 
quality  are  impaired  when  reducing  FR  from 

15  to  5  Hz.  Thus,  15  Hz  may  be  the  critical 
value  for  QoP. 

48 

Gulliver,  Serif, 

&  Ghinea, 

(2004) 

5,  15,  25  Hz 

Information 

assimilation 

QoP 

Low  FR  of  5  Hz  has  no  effect  on  enjoyment 
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Glossary  of  Acronyms 


2-D 

two-dimensional 

3-D 

three-dimensional 

AR 

augmented  reality 

FCS 

Future  Combat  System 

FR 

frame  rate 

GL 

gray  scale  level 

HMD 

head-mounted  display 

LOD 

level  of  detail 

MRI 

magnetic  resonance  imaging 

MSR 

mean  system  responsiveness 

QoP 

quality  of  perception 

QoS 

quality  of  service 

nns 

root  mean  square 

SD 

standard  deviation 

SDSR 

standard  deviation  of  system  responsiveness 

SL 

system  latency 

SR 

system  responsiveness 

UGV 

unmanned  ground  vehicle 

VE 

virtual  environment 

VR 

virtual  reality 

37 


NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  DEFENSE  TECHNICAL 
(PDF  INFORMATION  CTR 
ONLY)  DTICOCA 

8725  JOHN  J  KINGMAN  RD 
STE  0944 

FORT  BELVOIR  VA  22060-6218 

1  US  ARMY  RSRCH  DEV  &  ENGRG  CMD 
SYSTEMS  OF  SYSTEMS 
INTEGRATION 
AMSRD  SS  T 
6000  6TH  ST  STE  100 
FORT  BELVOIR  VA  22060-5608 

1  INST  FOR  ADVNCD  TCHNLGY 
THE  UNIV  OF  TEXAS  AT  AUSTIN 
3925  W  BRAKER  LN  STE  400 
AUSTIN  TX  78759-5316 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
IMNE  ALC  IMS 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  TL 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

2  DIRECTOR 

US  ARMY  RESEARCH  LAB 
AMSRD  ARL  Cl  OK  T 
2800  POWDER  MILL  RD 
ADELPHI  MD  20783-1197 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  M  DR  M  STRUB 
6359  WALKER  LANE  SUITE  100 
ALEXANDRIA  VA  22310 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MA  J  MARTIN 
MYER  CENTER  RM  2D3 1 1 
FT  MONMOUTH  NJ  07703-5630 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MC  A  DAVISON 
320  MANSCEN  LOOP  STE  166 
FT  LEONARD  WOOD  MO  65473-8929 


1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MD  T  COOK 
BLDG  5400  RM  C242 
REDSTONE  ARSENAL  AL  35898-7290 

1  COMMANDANT  USAADASCH 
ATTN  ATS  A  CD 

ATTN  AMSRD  ARL  HR  ME  MS  A  MARES 
5800  CARTER  RD 
FT  BLISS  TX  79916-3802 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MI  J  MINNINGER 

BLDG  5400  RM  C242 

REDSTONE  ARSENAL  AL  35898-7290 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MM  DR  V  RICE 
BLDG  4011  RM  217 
1750  GREELEY  RD 
FT  SAM  HOUSTON  TX  78234-5094 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MG  R  SPINE 
BUILDING  333 

PICATINNY  ARSENAL  NJ  07806-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MH  C  BURNS 
BLDG  1002  ROOM  117 
1ST  CAVALRY  REGIMENT  RD 
FT  KNOX  KY  40121 

1  ARMY  RSCH  LABORATORY  -  HRED 

AVNC  FIELD  ELEMENT 
ATTN  AMSRD  ARL  HR  MJ  D  DURBIN 
BLDG  4506  (DCD)  RM  107 
FT  RUCKER  AL  36362-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MK  MR  J  REINHART 
10125  KINGMAN  RD 
FT  BELVOIR  VA  22060-5828 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MV  HQ  USAOTC 
S  MIDDLEBROOKS 
91012  STATION  AVE  ROOM  111 
FT  HOOD  TX  76544-5073 

5  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MY 
2520  HEALY  AVE  STE  1 172  BLDG  51005 
FT  HUACHUCA  AZ  85613-7069 


38 


NO.  OF 

COPIES  ORGANIZATION 


NO.  OF 

COPIES  ORGANIZATION 


1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MP  D  UNGV ARSKY 
BATTLE  CMD  BATTLE  LAB 
415  SHERMAN  AVE  UNIT  3 
FT  LEAVENWORTH  KS  66027-2326 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  M  DR  B  KNAPP 
ARMY  G1  MANPRINT  DAPE  MR 
300  ARMY  PENTAGON  ROOM  2C489 
WASHINGTON  DC  20310-0300 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MJK  MS  D  BARNETTE 
JFCOM  JOINT  EXPERIMENTATION  J9 
JOINT  FUTURES  LAB 
115  LAKEVIEW  PARKWAY  SUITE  B 
SUFFOLK  VA  23435 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MQ  M  R  FLETCHER 
US  ARMY  SBCCOM  NATICK  SOLDIER  CTR 
AMSRD  NSC  SSE  BLDG  3  RM  341 
NATICK  MA  01760-5020 

10  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MT  DR  J  CHEN 
12350  RESEARCH  PARKWAY 
ORLANDO  FL  32826-3276 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MS  MR  C  MANASCO 
SIGNAL  TOWERS  RM  3  03 A 
FORT  GORDON  GA  30905-5233 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MU  M  SINGAPORE 
6501  E  11  MILE  RD  MAIL  STOP  284 
BLDG  200A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MF  MR  C  HERNANDEZ 
BLDG  3040  RM  220 
FORT  SILL  OK  73503-5600 

1  ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 


1  ARMY  RSCH  LABORATORY  -  HRED 

ATTN  AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 

1  ARL-HRED  LIAISON 

PHYSICAL  SCIENCES  LAB 
PO  BOX  30002 

LAS  CRUCES  NM  88003-8002 
1  DIRECTOR 

UNIT  OF  ACTION  MANEUVER  BATTLE  LAB 
ATTN  ATZKUA 
BLDG  1101 

FORT  KNOX  KY  40121 

1  DIR  FOR  PERS  TECHNOLOGIES 

DPY  CHIEF  OF  STAFF  PERS 
300  ARMY  PENTAGON  2C733 
WASHINGTON  DC  20310-0300 

1  CODE  1142PS 

OFC  OF  NAVAL  RSCH 
800  N  QUINCY  STREET 
ARLINGTON  VA  22217-5000 

1  CDR 

USA  AEROMEDICAL  RSCH  LAB 

ATTN  LIBRARY 

FORT  RUCKER  AL  36362-5292 

1  US  ARMY  NATICK  RD&E  CTR 
ATTN  STRNCYBA 
NATICK  MA  01760-5020 

1  PEO  STRI 

12350  RSCH  PARKWAY 
ORLANDO  FL  32826-3276 

1  GOVT  PUBLICATIONS  LIB 
409  WILSON  M 

UNIVERSITY  OF  MINNESOTA 
MINNEAPOLIS  MN  55455 

1  HUMAN  FACTORS  ENG  PROGRAM 
DEPT  OF  BIOMEDICAL  ENGNG 
COLLEGE  OF  ENGINEERING  & 

COMPUTER  SCIENCE 
WRIGHT  STATE  UNIVERSITY 
DAYTON  OH  45435 


39 


NO.  OF 

COPIES  ORGANIZATION 


1  DIR  AMC-FIELD  ASSIST  IN 
SCIENCE  &  TECHNOLOGY 
ATTN  AMC-FAST 
FT  BELVOIR  VA  22060-5606 

1  ENGINEERING  PSYCH  LAB 
DEPT  OF  BEHAVIORAL 

SCIENCES  &  LEADERSHIP 
BLDG  601  ROOM  281 
US  MILITARY  ACADEMY 
WEST  POINT  NY  10996-1784 

ABERDEEN  PROVING  GROUND 


1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  Cl  OK  (TECH  LIB) 
BLDG  4600 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  Cl  OK  S  FOPP1ANO 
BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  HR  MR  F  PARAGALLO 

BLDG  459 


40 


